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What Is NAEP? 



T he National Assessment of Educational Progress (NAEP) is a congres- 
sionally mandated project of the U.S. Department of Education’s Na- 
tional Center for Education Statistics. It assesses what U.S. students 
should know and he able to do in geography, reading, writing, mathematics, 
science, U.S. history, the arts, civics, and other academic subjects. Since 
1969, NAEP has surveyed the achievement of students at ages 9, 13, and 17 
and, since the 1980s, in grades 4, 8, and 12. Measuring educational achieve- 
ment trends over time is critical to measuring progress toward the National 
Education Goals. 



The National Assessment Governing Board 



The National Assessment Governing Board (NAGB) was created by Con- 
gress to formulate policy for the National Assessment of Educational Progress 
(NAEP). Among the Board’s responsibilities are developing objectives and 
test specifications, and designing the assessment methodology for NAEP. 
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Introduction 



T his document contains the design and rationale for assessing 
science achievement of students throughout the United States 
in 2005. This same Framework has been used hy NAEP for 
the 1996 and 2000 science assessments. The Framework provides a 
general overview of NAEP, describes the NAEP Science Eramework 
adopted by NAGB, and reviews the process by which the Eramework 
was developed. 

Background 

NAEP is authorized by Congress and funded by the federal gov- 
ernment; it is the only nationally representative and continuing as- 
sessment of what America’s students know and can do. Eor more 
than 35 years, NAEP has been charged with collecting and reporting 
information on student achievement in mathematics, reading, science, 
U.S. history, writing, and other subjects. Originally, assessments 
were of students at ages 9, 13, and 17, but beginning in 1983, they 
also included students in grades 4, 8, and 12. 

NAEP reports provide descriptive information about student per- 
formance, including basic and higher order skills, in various subjects 
and comparisons of performance by race/ethnicity, gender, type of 
community, and geographic region. NAEP data also show relation- 
ships between achievement and certain background variables, such 
as the time spent on homework or the educational level of parents. 

In the past, only results from a national sample of students were 
reported for each NAEP subject assessed, but, in 1987, a national 
study group chaired by Eamar Alexander, then governor of Ten- 
nessee, and H. Thomas James, president emeritus of the Spencer 
Eoundation, recommended to the Secretary of Education that NAEP 
collect representative data on student achievement at the state level. 
In 1990, a trial state assessment was conducted for eighth-grade 
mathematics. State-level assessments since have become a major 
part of NAEP in reading, writing, mathematics, and science. In 
2002, NAEP conducted the first Trial Urban District Assessment 
(TUDA) in nine urban districts that volunteered to participate. The 
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NAEP TUDA component has continued in subsequent years in 
reading and mathematics. 

Science educators, by and large, do not quarrel with the essential 
concept of assessment, but there has been no formal agreement on 
a common framework, outcomes, or goals and objectives to assess. 
The NAEP Science Assessment attempts to reflect a comprehensive, 
contemporary view of science so that those affected by the Nation- 
al Assessment are satisfied that it addresses the complex issues in 
science education without oversimplification. The framers of this 
document have attempted to tread a fine line between clear com- 
munication and technical accuracy in the hope that their efforts 
represent a step forward in building national consensus on the key 
outcomes of science education. 

Development of the NAEP Science Assessment 
Framework 

The following factors guided the process for developing consen- 
sus on the NAEP Science Assessment Framework: 

The first factor is the general process of consensus develop- 
ment, both as it is set forth in law and as it evolves over time. 

The process calls for active participation and broad involvement of 
curriculum specialists; science teachers; local science supervisors; 
state supervisors; administrators; parents; representatives of scien- 
tific associations, business and industry, government, and unions; 
cognitive psychologists; and science educators as well as participa- 
tion from the public and private education sectors. 

The second factor is emphasis on the important outcomes of 
science education. As much as possible, this Eramework represents 
what is considered essential learning in science, and set the stage 
for the 1996 and subsequent assessments. The Eramework also rec- 
ommends innovative assessment techniques to probe critical abili- 
ties and content areas. 

The third factor is recognition that the various “players” in 
education and industry often hold diverse, and sometimes con- 
flicting, views. Eurther, research and general agreement in the field 
is lacking. This lack of agreement on a common scope of instruction 
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and sequence, the components of scientific literacy, the important 
outcomes of learning, and the nature of overarching themes in sci- 
ence hinders clear communication between science educators and 
the public. 

The process of developing the Framework document and accom- 
panying reports occurred between October 1990 and August 1991. 
Original plans called for a new NAEP Science Assessment to be 
given in 1994. Due to a budget shortfall, however, both the new 
science and mathematics assessments were rescheduled for 1996. 

For the consensus process, a steering committee of 1 8 members 
(see appendix E) recommended by the education community and 
the private sector was established. Its members developed the princi- 
ples that guided the creation of the Eramework. A smaller planning 
committee (see appendix E), composed of practitioners, recognized 
experts in science education, and scientists, was established to iden- 
tify goals and objectives and to produce the Eramework. Together 
with the steering committee, it was also responsible for suggesting 
ways to assess important outcomes of science education. 

The planning committee met monthly from November 1990 
through April 1991 and was joined in its first and final meetings by 
the steering committee, which reviewed and reacted to all Erame- 
work drafts. Staff of the American Institutes for Research (AIR), a 
subcontractor, also attended the meetings. On the basis of this input 
and reaction and advice from the committees, AIR formulated 
specifications for the science assessment. 

NAGB Subject Area Committee #2 and technical staff closely 
monitored the consensus project work and Board members were 
involved in all key phases. In addition, advice was sought from the 
organizations and sectors that affect and are affected by science 
education. Eor instance: 

• Opportunities for public input were provided at two hearings 
(in Washington, D.C., and in San Erancisco). Opinions were 
expressed by representatives of institutions (for example, pub- 
lic and private education, scientific and science education 
organizations) likely to be affected by or concerned with the 
Eramework and subsequent assessments. 
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• The Council of State Science Supervisors was kept abreast of 
project developments through its electronic communication 
system, PSInet. Input from the Council of State Science Su- 
pervisors was particularly important in helping the NAEP 
Science Assessment Project define “big ideas” or Themes in 
science learning (see chapter 2). For example: 

— States with existing frameworks for science instruction 
were contacted for copies of their frameworks and assess- 
ment methods. 

— Planning and steering committee members hosted sessions 
at both regional and national science education meetings 
to report on the development of the NAEP Science Assess- 
ment Framework. 

— The revised draft of the Framework was widely circulated 
in June 1991 within the science education and science 
communities for reaction and comments. 

Steering Committee Guidelines 

At its early meeting, the steering committee drafted guidelines 
for the planning committee’s work and recommended that the 
Framework and ensuing NAEP Science Assessment have the fol- 
lowing five characteristics: 

• The Framework should reflect the best thinking about the 
knowledge, skills, and competencies needed for a high degree 
of scientific understanding among all students in the United 
States. Accordingly, it should: 

— encompass knowledge and use of organized factual infor- 
mation, relationships among concepts, major ideas unify- 
ing the sciences, and thinking and laboratory skills; 

— be based on current understandings from research on 
teaching, learning, and student performance in science. 
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• Both the Framework and the NAEP Science Assessment 

should: 

— address the nature and practices of knowing in science as 
they differ from other ways of knowing; 

— reflect the quantitative aspects of science as well as the 
concepts of life, Earth, and physical sciences; 

— deal with issues raised hy the role of science and technolo- 
gy in society; 

— include practical problemsolving that involves design, use 
of materials, and weighing risks in relation to benefits; 

— take into account the developmental levels of students; 

— ensure that students with diverse backgrounds are assessed 
in ways that provide them with equal and fair opportunities 
to reflect their knowledge and performance. 

• Assessment formats should be used that are consistent with 

the objectives being assessed. A variety of strategies for as- 
sessing student performance are advocated, including: 

— performance tasks that allow students to manipulate phys- 
ical objects and draw scientific understandings from the 
materials before them; 

— open-ended items that provide insights into students’ levels 
of understanding and ability to communicate in the sci- 
ences, as well as their ability to generate, rather than sim- 
ply recognize, information related to scientific concepts 
and their interconnections; 

— collections of student work over time (such as portfolios) 
that demonstrate what students can achieve outside the 
time constraints of a standardized assessment situation; 

— multiple-choice items that probe students’ conceptual un- 
derstanding and ability to connect ideas in a scientifically 
sound way. 
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• The assessment should contain a hroad enough range of items 
at different levels of proficiency to identify three achievement 
levels for each grade. 

• Information on students’ demographic and other background 
characteristics should he collected. Additional information 
should he collected from students, teachers, and administra- 
tors about instructional programs and delivery systems so that 
their relationships with student achievement can be ascer- 
tained and used to inform program and policy decisions. 
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Chapter One 

The Nature of Science and the 
Science Curriculum 



The Nature of Science 

T he various fields of science all have their own special ways 
of knowing. However, for the purpose of planning appropriate 
assessments, the essentials of the natural sciences should he 
defined. 

The natural sciences are characterized hy organized explanations 
incorporating both theoretical and empirical elements. Scientists at- 
tempt to construct theories that encompass as much factual and 
conceptual knowledge, including laws and principles, as possible. 
The construction of theories is a process involving the consider- 
ation of factual evidence, insightful questioning, creativity, and 
imagination. 

Atomic theory and evolution theory are good examples of mod- 
ern scientific theories that are central to the sciences as ways of 
knowing. They are sources of new hypotheses and logical deduc- 
tions that can be tested. As these theories are refined, they continue 
to stimulate new questions and hypotheses. As with other scientific 
theories, each time they come into play in experimental situations, 
they are again subject to testing and the possibility of refutation. 
Each success broadens their domain and increases their usefulness. 
Verification of theories does not ensure truth, but it does extend 
their usefulness in explaining natural phenomena. 

Useful theories enable an individual to make predictions under a 
specified range of conditions. Scientific theories must be able to be 
tested according to standards of evidence and logical argumentation 
set by the scientific community; they are central to the scientific 
way of knowing and guiding observation. 
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When scientists observe natural phenomena, their observations 
are made within the context of existing scientific theories. Nonsci- 
entists holding different views may perceive the same phenomena, 
but through their subjective belief systems, may arrive at different 
conclusions. History is replete with examples of misunderstandings 
and miscommunication that are based, at least in part, on the use of 
different rules of observation. 

Science consists of both theoretical and experimental knowledge 
that is constructed by the creativity, knowledge, and world view of 
scientists. Experimentation plays an essential role in generating and 
verifying scientific information. New findings and results of scien- 
tific observation and experimentation accumulate, enlarging the base 
of present knowledge and laying the foundation for future learning. 
Usually, scientific knowledge is structured, forming a web of inter- 
related concepts, laws, and principles. Whereas sensory data are 
sometimes ephemeral, concepts are precisely formulated and are 
connected to other concepts or sensory data within a theory in mul- 
tiple ways. Concepts without factual content (sensory data) are 
empty, whereas sensory data without concepts are difficult to un- 
derstand and open to misinterpretation. 

None of the processes usually associated with science (for ex- 
ample, observing, measuring, classifying, deducing, inferring) is 
unique to science. However, in science, these processes are given 
meaning by the context of the subject matter under investigation. 
Hence, observations of a mealworm seemingly walking endlessly 
around the sides of a container have new meaning when mealworm 
anatomy, behavior, and physiology are understood. Likewise, the 
wiggly lines a physicist observes on a photographic negative of 
a cloud chamber remain wiggly lines to most observers, but they 
become important sensory data (facts) when the conditions under 
which the lines were produced are known. A goal of science educa- 
tion, therefore, is to help students recognize the difference between 
personal opinion and knowledge gained through scientific investi- 
gation and debate. 

Although the scientific disciplines are alike in their reliance on 
evidence, use of logic, and organization of factual information into 
concepts and theories, they differ with respect to what constitutes 




evidence, specific methods of investigation, and degree of quantifi- 
cation. And yet, “there are common understandings among [scien- 
tists] about what constitutes an investigation that is scientifically 
valid” (American Association for the Advancement of Science, 
1989). This view of the nature of science has profound implications 
for assessment, as reflected in the criteria outlined in chapter 3. 

The Science Curriculum 

Because a major purpose of NAEP is to illuminate education poli- 
cy, assessment of student science learning must take into account the 
science curriculum. Hence, the NAEP Science Assessment Frame- 
work is based on a consensus regarding desirable elements of sci- 
ence education against which student attainment is to be measured. 

In developing the Framework, the planning committee reviewed 
key blue-ribbon reports, examined exemplary practices, studied 
local and state-based innovations in science curricula, reviewed sci- 
ence education literature, and noted innovations emerging in other 
countries. Recent reports by government agencies and professional 
societies (National Science Board Commission on Precollege Edu- 
cation in Mathematics, Science and Technology, 1983; American 
Association for the Advancement of Science, 1989; National Science 
Teachers Association, 1989; National Research Council, 1990) ex- 
press unanimity in their goals for science education. For example: 

• Students should acquire a core of scientific understanding, 
including organized factual information. 

• Students should acquire the ability to relate one scientific 
concept to others and to problems they encounter in and out 
of school. 

• Students should be able to apply science knowledge in practi- 
cal ways. 

• Students should be familiar with experimental design and 
have the ability to carry out scientific experiments that are 
developmentally appropriate. 
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• Students should acquire the science knowledge and understand- 
ing that will allow them the opportunity to pursue further 
study in scientific fields or enter science- or technology- 
related careers. 

There also are similarities in the reports’ recommendations on 
the science curricula and instruction needed to achieve these goals, 
including: 

• reduction of the traditional breadth of coverage in favor of 
greater depth, especially in high school science; 

• emphasis on development of such thinking processes as or- 
ganizing factual knowledge around major concepts, defining 
and solving problems, accessing information and reasoning 
with it, and communicating with others about one’s science 
results and understandings; 

• multidisciplinary and interdisciplinary approaches to science 
teaching; 

• approaches that encourage active student involvement and par- 
ticipation, such as participating in hands-on science activities; 
learning in small, cooperative groups; reflecting orally and in 
writing on experiences; and completing sustained projects; 

• increased participation of underrepresented populations in 
challenging school science. 

The design of the NAEP Science Assessment Framework, although 
maintaining some conceptual continuity with the 1990 NAEP Sci- 
ence Assessment, takes into account current reforms in science ed- 
ucation. It is also consistent with the science framework used for 
the 1991 International Assessment of Educational Progress and the 
science framework for the 1995 International Association for the 
Evaluation of Educational Achievement (lEA) assessments. 
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Chapter Two 

The Framework for the 
NAEP Science Assessment 



Introduction 

I t is customary to collect relevant curriculum guides, frameworks, 
and other course outlines to get a sense of what students are 
studying throughout the nation when developing a framework for 
an educational assessment. The union of all these “learning guides” 
is used to develop overall test specifications. Such an approach tends 
to have an unfortunate consequence: it often leads to a hroad, trivi- 
alized, lowest common denominator approach to assessment. How- 
ever, these materials can serve another purpose, that of documenting 
trends and developments in science education throughout the country. 
In the NAEP Science Assessment Framework development process, 
science reform reports from states, in addition to several large-city 
science curriculum guides, were gathered and used to establish con- 
sonance between the evolving Framework and the most forward- 
looking of the reports and guides. 

The traditional approach to teaching science tends to emphasize 
rote memorization of facts without connection or organization. Al- 
though it must not lose sight of the need for factual knowledge that 
is fundamental to science literacy, assessment of science achievement 
must change to give more emphasis to conceptual understanding 
and the application of knowledge and skills for several compelling 
reasons. First, the expansion of scientific information has resulted 
in far too many facts for a student to memorize. It is more efficient 
to store them electronically (or in other forms) and access informa- 
tion as needed. Second, isolated science facts that are not organized 
or used tend to be forgotten quickly and, even when remembered, 
form a poor basis for learning. Third, it is desirable to encourage 
science instruction that is used both to deepen understanding and 
to address challenging problems. Science education is best served 
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when students can understand and discuss ideas rather than simply 
accumulate unconnected facts. 

Therefore, it is important that science assessment covers major 
topics like electricity and magnetism, forces and motions, life cy- 
cles, ecosystems, plate tectonics, and climatology. However, even 
these topics need to he viewed as shorthand for a much richer un- 
derstanding of what students should attain. Several current reform 
reports and frameworks, such as Science for All Americans: A Proj- 
ect 2061 Report on Literacy Goals in Science, Mathematics, and 
Technology (American Association for the Advancement of Science, 
1989), innovative state frameworks, and reports from the National 
Center for Improving Science Education (1990, 1991) describe de- 
sired outcomes of science instruction in new ways. They advocate 
mastery of fundamentals in ways that are more likely to result in 
students’ learning. The approach advocated in these documents is 
reinforced hy the findings of science educators and cognitive re- 
searchers, demonstrating that if students do not completely learn 
the concepts presented to them, they pass through the K-12 grades 
without fundamentally changing the conceptual models they 
learned in their early years. 

Whereas it is easy to argue for a test that emphasizes conceptual 
understanding rather than topical listings and recognition of defini- 
tions from a list of choices, it is more difficult to agree on how sci- 
ence learning should he assessed. The science courses that were 
developed in the late 1950s and early 1960s used a more conceptual 
organization for textbooks and teaching materials. These programs, 
funded by the National Science Foundation, became major influ- 
ences on the country’s secondary science textbooks from 1960 to 
1970 and helped raise standards of students’ science achievement 
(Shymansky et ah, 1983). Unfortunately, old-style textbooks have 
since returned, with expanded numbers of pages that include many 
newly emerging topics. The effect on students’ understanding of 
science has not been beneficial. The NAEP Science Assessment 
Framework incorporates a balance of knowledge and skills based 
on current reform reports, exemplary curriculum guides, and re- 
search on the teaching and learning of science. 
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Framework Elements 

The Framework for the NAEP Science Assessment is organized 
according to two major dimensions: Fields of Science and Knowing 
and Doing Science. The Fields of Science are Earth, physical, and 
life sciences. Knowing and Doing Science includes Conceptual 
Understanding, Scientific Investigation, and Practical Reason- 
ing. These two dimensions and their suhdimensions are explained 
in greater detail helow and in appendix B. 

The Matrix 

The NAEP Science Assessment Framework is structured ac- 
cording to a matrix similar to that used for the 1990 NAEP Science 
Assessment. The content areas are organized into the same three 
fields; however, an additional requirement for some interdiscipli- 
nary exercises merges technology with the science content areas. 
The Nature of Science (which also was part of the 1990 Erame- 
work) and Themes are categories that should integrate the three 
fields of science, rather than represent separate content. The 
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Knowing and Doing dimension is a reorganization of the Think- 
ing Skills dimension that was a part of the 1990 NAEP Science 
Assessment, with a clearer delineation of suhcategories, particular- 
ly with respect to Practical Reasoning. Each element of the NAEP 
Science Assessment Framework is addressed briefly helow and in 
greater detail in separate appendixes. 

With respect to Fields of Science, the main emphasis of the 
assessment should he on knowledge in the content areas. Details 
follow: 

• Distribution of content across the three science fields should 
be approximately equal in grades 4 and 12. 

• Eor grade 8, the Eramework places a somewhat heavier em- 
phasis on life science (40 percent), with physical and Earth 
sciences distributed equally (30 percent each). The distribu- 
tion for grade 8 reflects the importance of human biology, 
which gets increased recognition in both curriculum and in- 
struction, for this age group. 

• A limited number of exercises at every grade level should 
address technology and its relationship to science. Although 
every item need not do so, exercises and tasks that draw from 
more than one discipline at the same time are highly desir- 
able, as they are more likely to mirror science problems that 
occur in the real world. 

More specific guidance on the distribution of content from the 
three fields is given in the Specifications document, which is used 
by test developers to create the assessment. 

A major emphasis with respect to Knowing and Doing Science 
should be on students’ active expression of conceptual understand- 
ing. Details follow: 

• At each grade level, 45 percent of content should be devoted 
to Conceptual Understanding, or the ability to understand 
basic concepts and tools used in the process of a scientific 
investigation. 
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• Scientific Investigation, which is the ability to use the appro- 
priate tools and thinking processes in the application of sci- 
ence, should he more heavily emphasized in grade 4 (45 
percent) than in grades 8 and 12 (30 percent at each of these 
grade levels). This is desirable because learning by doing is 
crucial for younger students, and ways of knowing in science 
need to be introduced early. 

• Practical Reasoning involves suggesting effective solutions 
to everyday problems by applying scientific knowledge and 
using skills. The ability to engage in practical reasoning is es- 
sential if a student is to solve complex problems or apply pre- 
vious knowledge to an everyday problem. The proportion of 
practical reasoning questions should be 10 percent in grade 4 
and 25 percent in grades 8 and 12. 

• Many exercises will involve more than one of the subdimen- 
sions in Knowing and Doing Science. 

• The percentages cited above should not be interpreted as im- 
mutable; they should serve as a general guide for test devel- 
opment. As new assessment tasks are developed, assessment 
of conceptual understanding may become a part of exercises 
that measure scientific investigations and practical reasoning. 

More detailed guidelines on the distribution of Knowing and 
Doing items are provided in the Specifications document. 

Every question or task in the assessment should be classifiable 
by one or more subcategories in each of the two major dimensions 
of the matrix. In addition to the two major dimensions, the Frame- 
work includes two other categories that pertain to a limited subset 
of items. The first concerns students’ understanding of the Nature 
of Science: 

• This category includes the history of science and technology, 
the habits of mind that characterize these fields, and methods 
of inquiry and problemsolving. 

• At least 15 percent of the content should measure the Nature 
of Science. Within this segment, somewhat more than half 
(about 60 percent) should deal with the nature of science and 
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somewhat less than half (about 40 percent) with the nature of 
technology. 

In specifying these percentages, it is assumed that assessment 
items can he developed to measure knowledge of content within a 
field of science or an area of Knowing and Doing Science, in ad- 
dition to measuring knowledge of the Nature of Science. 

The second category, Themes, represents hig ideas or key or- 
ganizing concepts that pervade science education. Themes cross 
traditional science discipline boundaries and make up the inquiry 
tools that scientists use to better investigate and understand phe- 
nomena. These themes include the notion of Systems and their 
application in the disciplines; Models and their function in the de- 
velopment of scientific understanding and application to practical 
problems; and Patterns of Change as they are exemplified in natu- 
ral phenomena. Guidelines follow: 

• The assessment should probe, in a developmentally appropri- 
ate way, students’ understanding of these themes. 

• Students in grade 4 should build beginning notions related to 
systems, models, and patterns of change; about one-third of 
the assessment (spread evenly across the three themes) should 
measure themes as well as content from one or more of the 
fields of science. 

• Fifty percent of the assessment content in grades 8 and 12 
should assess students’ understanding of the themes; ques- 
tions should be distributed evenly among the three themes. 

• The assessment exercises must address a specific theme with- 
in a science content area so that an understanding of the con- 
tent and the theme are probed at the same time. 

More detailed guidelines on assessing themes are provided in 
the Specifications document. 

Understanding, doing, and using science often involve tasks that 
include more than one category in each dimension. Multiple-duty 
exercises may present some scoring challenges. Relatively simple 
exercises will be scorable according to one or two subdimensions 
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that include, for example, a “conceptual understanding in the phys- 
ical sciences.” More complex items may he scorahle in several suh- 
categories; for example, an open-ended task involving ecosystems 
might yield responses scorahle according to “conceptual under- 
standing in the Earth sciences,” “scientific investigation in the life 
sciences,” and “systems.” Such items, which contrihute to more 
than one suhcategory, may prove difficult to reproduce through 
similar items having the same properties in future assessments. It is 
important, therefore, to specify carefully the several suhdimensions 
or suhcategories that each such exercise is intended to prohe. Scor- 
ing rubrics for each of the suhdimensions also must he developed. 

The Fields of Science 

The descriptions helow are summaries of the major topic areas 
to he prohed within each field in the NAEP Science Assessment. 
This content represents key elements in science that all students 
should he expected to know and understand. Eor a complete expla- 
nation, including descriptions of subject matter knowledge to be 
expected at each grade level, see appendix A. 

Earth Science 

The NAEP Science Assessment will probe student understanding 
of how Earth scientists depict data through maps and other means to 
interpret objects, their features and structures, and the events and 
processes that caused them. What do students know about their own 
position with respect to objects and structures on, below, and above 
the Earth’s surface? What do students know about the changes in 
position of objects and environments through time? What do stu- 
dents know about the relative movements of the Earth, the Moon, the 
Sun, and the planets? The content to be assessed in Earth science 
centers on objects and features that are relatively accessible or visi- 
ble: the solid Earth (lithosphere), water (hydrosphere), air (atmos- 
phere), and the Earth in space. With respect to Earth science, the 
NAEP Science Assessment should focus on the following concepts 
and topics. 

Concepts related to solid Earth: 

• composition of the Earth 



17 




• forces that alter the Earth’s surface 

• rocks and their formation, characteristics, and uses 

• soil and its changes and uses 

• natural resources used hy humankind 

• forces within the Earth 
Concepts related to water: 

• water cycle 

• nature of the oceans and their effects on water and climate 

• location, distribution, and characteristics of water, and its 
effect and influence on human activity 

Concepts related to air: 

• composition and structure of the atmosphere, including energy 
transfer 

• nature of weather 

• common weather hazards 

• air quality and climate 
Concepts related to Earth in space: 

• setting of the Earth in the solar system 

• setting and evolution of the solar system in the universe 

• tools and technology used to gather information about space 

• apparent daily motions of the Sun, the Moon, the planets, and 
the stars 

• rotation of the Earth about its axis and the Earth’s revolution 
around the Sun 

• tilt of the Earth’s axis that produces seasonal variations in 
climate 

• Earth as a unique member of the solar system (may be ap- 
proximated in other galaxies in the universe) that evolved at 
least 4.5 billion years ago 
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Physical Science 

The physical science component of the NAEP Science Assess- 
ment relates to basic knowledge and understanding concerning the 
structure of the universe as well as the physical principles that op- 
erate within it. The assessment should prohe the following major 
topics: matter and its transformations, energy and its transforma- 
tions, and the motion of things. The NAEP Science Assessment 
should focus on the following physical science concepts. 

Matter and its transformations: 

• diversity of materials, including the classification, types, and 
particulate nature of matter 

• temperature and states of matter 

• properties and uses of material, including modifying proper- 
ties and the synthesis of materials with new properties 

• resource management 
Energy and its transformations: 

• forms of energy 

• energy transformations in living systems, natural physical 
systems, and artificial systems constructed hy humans 

• energy sources and use, including distribution, energy conver- 
sion, and energy costs and depletion 

Motion: 

• an understanding of frames of reference 

• force and changes in position and motion 

• action and reaction 

• vibrations and waves as motion 

• general wave behavior 

• electromagnetic radiation 

• interactions of electromagnetic radiation with matter 
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Life Science 

The fundamental goal of life science is to attempt to understand 
and explain the nature and function of living things. During the 
20th century, the focus of biological research has changed from de- 
scriptive natural history to experimental investigation, with evolu- 
tion as the central, unifying theory. The following list contains the 
major concepts to he assessed in life science. 

Change and evolution: 

• diversity of life on Earth 

• genetic variation within a species 

• theories of adaptation and natural selection 

• changes in diversity over time 
Cells and their functions: 

• information transfer 

— energy transfer for the construction of proteins 
— communication among cells 
Organisms: 

• reproduction, growth, and development 

• life cycles 

• functions and interactions of systems within organisms 
Ecology: 

• interdependence of life (populations, communities, and 
ecosystems) 

Knowing and Doing Science 

In the 1990 NAEP Science Assessment, three categories were 
used: Knowing Science, Solving Problems, and Conducting In- 
quiries. Eor the current NAEP Science Assessment, it should he 
noted that not only has the Knowing Science dimension been 
changed, but its meaning has also been redefined as well. It has 
been reformulated as Conceptual Understanding of Science to 
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stress the connections and the organization of factual knowledge in 
science. 

The suhdimension Scientific Investigation has been substituted 
for the 1990 category Conducting Inquiries. This new subcatego- 
ry is intended to probe students’ abilities to use the tools of sci- 
ence, including both cognitive and laboratory tools. Appropriate to 
their age and grade level, students should be able to acquire new 
information, plan appropriate investigations, use a variety of scien- 
tific tools, and communicate the results of their investigations. 

The 1990 category Solving Problems proved to be ambiguous. 
To emphasize the need to assess students’ abilities to use and apply 
science understanding in new, real-world applications, the category 
has been redefined as Practical Reasoning. Practical reasoning 
subsumes competence in analyzing a problem, planning and evalu- 
ating appropriate approaches, carrying out the required procedures 
for the approach(es) selected, and evaluating the result(s). 

Each of the three categories is described below. 

Conceptual Understanding 

Mastery of basic scientific concepts can best be shown by a stu- 
dent’s ability to use information to conduct a scientific investiga- 
tion or engage in practical reasoning. Optimally, essential scientific 
concepts involve a variety of information, including: 

• facts and events learned from science instruction and through 
experiences with the natural environment; 

• scientific concepts, principles, laws, and theories that scien- 
tists use to explain and predict observations of the natural 
world; 

• information about procedures for conducting scientific 
inquiries; 

• information about procedures for the application of scientific 
knowledge in the engagement of practical tasks; 

• propositions about the nature, history, and philosophy of 
science; 



21 




• kinds of interactions between and among science, technology, 
and society. 

The goal of school science is to engender conceptual understand- 
ing. Students should acquire information in ways that will enable 
them to apply it efficiently in the design and execution of scientific 
investigations and in practical reasoning. 

A challenge in the design of assessment exercises is to capture 
changes in the characteristics of student performance as children 
mature. In the primary years, when the goal of school science is to 
build a rich collection of information derived from examined expe- 
riences with the natural environment, the assessment of conceptual 
understanding will focus on the breadth of information acquired 
about the natural world and the student’s ability to elaborate on 
principles by using personal experiences. Does the student know the 
cyclical changes in the apparent shape of the Moon over time? More 
importantly, can the student relate how he or she knows about the 
changes? What evidence does the assessment exercise provide that 
the student’s information is based on direct experience? Is there a 
science notebook in which the student recorded observations of the 
Moon over time? Does the student know that sometimes the Moon 
is visible during daylight hours? In the primary years, the focus 
should not be on explanation or prediction, but instead on knowledge 
obtained from rich experiences in school. Consequently, assessment 
exercises would not be concerned with having students explain why 
the Moon appears to change shape but rather with relationships be- 
tween time of day, apparent positions of the Sun and the Moon, 
and times of moonrise and sunset. 

In the middle and high school years, the emphasis should shift 
from richness of experience to reasonable scientific interpretation of 
observations. In the elementary years, the primary concern should 
be with how well reasoned the interpretation is presented by the 
student, not with whether it reflects the most sophisticated scientif- 
ic reasoning. However, at grades 8 and 12, the assessment should 
be increasingly concerned with the congruence of the students’ in- 
terpretations with accepted interpretations, as well as with the so- 
phistication of their reasoning in moving from observations of the 
natural world to explanations and predictions. Of special interest 
in the 2005 NAEP Science Assessment will be the extent to which 
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students are able to understand and use the notions of models, sys- 
tems, and patterns of change. 

It is important to note that many aspects of conceptual under- 
standing as defined for the new NAEP Science Assessment cannot 
be tested using exclusively multiple-choice items. Items of this kind 
may be satisfactory for assessing individual parts of the information 
base, but they are limited in tapping highly valued aspects of con- 
ceptual understanding. 

Scientific Investigation 

Scientific investigation represents the activities of science that 
distinguish it from other ways of knowing about the world. It incor- 
porates such previously used assessment categories as “processes of 
science” and “scientific problemsolving.” This category is not just 
another name for the scientific method. Indeed, there is great confu- 
sion about the scientific method in the teaching of science. Real sci- 
ence is doing what one can in any way one can, often creatively and 
insightfully and using flashes of insight with little regard for a pro- 
gression of steps. However, there is a familiar format and context 
for reporting the results of experiments. It begins with the report of 
the problem and continues with the hypothesis, the experimental de- 
sign, the data collected, the analysis of those data, and the conclu- 
sions (if any). This convention of science is often mistaken for how 
scientists actually work. The results must satisfy logical analysis, 
but logical ordering may appear only when the report is prepared. A 
great disservice has been done to generations of students because 
well-meaning people have taught the standard method of reporting 
science as the standard method of doing science. 

Scientific investigations must be designed at levels appropriate 
to the development of the students. This component has important 
implications for assessment. Young students are limited in their 
ability to perceive the scale of both very large and very small things. 
Students’ limitations handicap them when they are forced, either by 
the textbook or by the curriculum, to deal with developmentally in- 
appropriate concepts such as atoms or even cells. Young students 
are also developmentally limited in their ability to understand time. 
The distant past and the future are narrowly perceived by the ego- 
centric student. Instruction, as well as assessment, must recognize 
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where the student is and take developmental levels into account. As 
students develop and accumulate experiences, their performance in 
doing scientific investigations should begin to look more and more 
like “real” science. 

Central to the ways scientists work is the concern for a fair test, 
for a controlled experiment. Children seem to have an intuitive sense 
of what makes a fair test. What they lack is the ability to consider all 
the variables and the means to control the variables. It might be rea- 
sonable to consider a developmental continuum such as the follow- 
ing when thinking about control of variables: 

• The first level of variables contains the simplest type: the 
nominal variable. Nominal variables have two or more un- 
ordered values: “This plant was watered; that plant was not.” 
“This seed was placed in the sunlight and that one was placed 
in the dark.” 

• The second level of variables is the ordinal variables level. 
These variables have a sequential order and no determined 
intervals (for example, the sequential ordering of objects by 
relative weight). 

• The third level of variables is the continuous variables level. 
These variables have sequence and equal intervals and are 
on a continuous scale: “This object has a temperature of 

50 degrees Celsius and that object has a temperature of 57 
degrees Celsius.” 

• The fourth level of variables is the ratio variables level. These 
variables are similar to continuous variables but have an ab- 
solute beginning point (for example, Kelvin temperature scale 
with an absolute zero point). 

As students are asked to demonstrate their ability to do scientific 
investigations, it is important to keep in mind this sort of develop- 
ment in understanding and performance, not only with respect to 
the control of variables, but also regarding the other elements of 
doing science. The difficulty with the assessment may not be with 
the content, but with the level of variable embedded in the content. 
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Practical Reasoning 

Practical reasoning about matters with scientific content (that is, 
the ability to apply one’s knowledge, thought, and action to real 
situations, not textbook problems) is influenced by the ability to 
(1) abstract and consider hypothetical experiences, (2) consider 
several factors simultaneously, (3) take a depersonalized view, and 
(4) realize the importance of practical reasoning and life experi- 
ence. These factors develop throughout life. 

One of the characteristics of young children is that they have diffi- 
culty dealing with multiple ideas simultaneously. With maturity and 
experience, they can consider several ideas at once and weigh bene- 
fits in relation to costs or risks. Their ability to abstract and consider 
hypothetical situations develops as students progress in science and 
learn to deal with more remote phenomena and generalizations. 

As they mature, students also learn to take depersonalized views 
of situations and to consider other people’s points of view. Often, 
real-life problems involve not only theoretical and technical ele- 
ments, but also personal preferences. What will be the social im- 
pact of a new waste disposal system? What will neighbors say if a 
traffic light is installed? How will other students react if lunchroom 
noise is diminished by staggering the lunch hour? To consider 
these questions carefully, it is necessary to understand different 
perspectives. The ability to understand the viewpoints of others in- 
creases with age and experience. 

Young children also may not realize the need for scientific infor- 
mation in solving problems. For example, children below the age of 
12 usually see no need to carry out measurements (Strang, 1990). 
Also, because young children have little responsibility for decisions 
alfecting their fives, they may not see the need for practical reason- 
ing. However, the more that students have done or seen, the more 
likely it is that they can solve real-world problems. With age and ex- 
perience, the possibility increases that a new situation is analogous 
to a previous one and that the human, technical, and theoretical fac- 
tors involved in a new situation have already been encountered. 

All these factors suggest that practical reasoning should become 
a major factor in science assessment at grades 8 and 12 rather than 
at grade 4. As students become eager to take control of their fives. 
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wish to try out their understanding of the world, and progress in de- 
velopment, practical situations related to their everyday life, school, 
and home provide excellent exemplars to demonstrate science-related 
practical reasoning. Thus, students might he asked to discuss prob- 
lems such as noise abatement in the lunchroom, to design a simple 
apparatus such as a flashlight or a burglar alarm, or to plan a 
school garden. 

By grade 12, students should be able to discuss larger science- 
and technology-linked problems not directly related to their im- 
mediate experiences. Examples of these might be waste disposal, 
energy uses, air quality, water pollution, noise abatement, and the 
tradeoffs between the benefits and adverse consequences of various 
technologies. 

The Nature of Science 

Knowledge of the nature of science is central to the understand- 
ing of scientific enterprise. Yet often this category is relegated to a 
discussion (or even rote memorization) of some version of the sci- 
entific method. There is total agreement that this topic should play 
a prominent role in the NAEP Science Assessment. 

A controversy existed within the project committees concerning 
whether this category is sufficient unto itself, or whether the Erame- 
work should include a separate section that deals with the nature of 
technology. The project committees were split on this issue. How- 
ever, all acknowledged that technology is integral to the nature of 
science and ought to be included in the assessment, provided that it 
clearly does not exist as a separate subcategory within the assess- 
ment. Technology, then, will be measured as it relates to science 
and the scientific enterprise. 

Science 

The following concepts are appropriate for assessment at the 
given levels: 

• By grade 4, students should understand that science is trying 
to find out what happens in the natural world. Through care- 
ful observation of objects and events, they should be able to 
develop explanations for their observations. Students should 
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also understand that different people may notice different 
things, and therefore may explain things differently. 

• By grade 8, students should have acquired an understanding 
of the control of variables and the difference between show- 
ing that conditions occur together and that they are causally 
related. Students should grasp what makes for a good scien- 
tific explanation by using all the relevant observations; sug- 
gesting what new observations to make; and explaining, as 
simply as possible, a wide variety of observations. 

• By grade 12, students should demonstrate their knowledge 
and understanding of the following: 

— Scientific conclusions are based on logic and evidence, but 
no fixed series of steps make up a scientific method. Scien- 
tists try to invent explanations that are logical and that fit 
observations, but these are subject to change based on new 
evidence. 

— Explanations are most believable when they also account 
for observations that were not made by the explainer. 

— Scientists (like anyone else) tend to look for, pay attention 
to, and cite evidence that supports what they already believe. 

— New conclusions require that scientists consider all possi- 
ble objections to their own findings. 

— Scientific organizations try to avoid bias and maintain quali- 
ty by having scientists’ reports of observations and explana- 
tions judged by other scientists before they are published. 

— Few human problems can be solved with scientific knowl- 
edge alone. Most are too complicated and involve values, 
about which science has little to say. 

Technology 

Students are surrounded by and interact with the manmade world 
as much as with the natural world. Therefore, they must develop an 
understanding of what shapes the design and development of the 
technologies that are a part of that manmade world and their daily 
lives. Rather than being a content area, technology is embedded 
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within this section because of its close association with science. 

The following concepts are appropriate for assessment at the given 
levels: 

• By grade 4, students should understand that any design re- 
quires making tradeoffs and that advantages and disadvan- 
tages must he weighed. 

• By grade 8, students should understand that scientific knowl- 
edge is often useful in design and that much scientific inves- 
tigation is done for the purpose of improving design. They 
should understand that there are often several ways to solve a 
design problem and that possible solutions should be evalu- 
ated on, and justified by, their advantages and disadvantages. 

• By grade 12, students should know that scientific knowledge 
may help to predict consequences of one design or another, 
but that design decisions often depend on human values that 
are outside of science. They should also be able to apply sci- 
entific concepts to scientific, societal, and/or technical con- 
cerns. They should understand that every design has limits 
and may fail if it is required to work outside of these limits. 

Themes 

Themes are the “big ideas” of science that transcend the various 
scientific disciplines and enable students to consider problems with 
global implications. To understand the conceptual basis for the 
themes that have been selected, students must begin to develop an 
understanding of major ideas by the fourth grade. They should con- 
tinue to develop their understanding through the 8th grade, and by 
the 12th grade, they should have the ability to integrate their 
knowledge and understanding. 

The review of current state frameworks conducted in the course 
of developing the new NAEP Science Assessment Framework re- 
vealed that many are based in part on crosscutting themes in science. 
Several national organizations, including the American Association 
for the Advancement of Science, have issued reports that advocate 
the importance of common themes. The number of themes defined 
in these reports and state frameworks varies somewhat, but there is 
considerable agreement on which common elements or big ideas of 
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science should be understood by students as they complete their 
high school education. The decision by the NAEP Science Assess- 
ment committees to include themes underscores their emerging im- 
portance, as well as the necessity of integrating themes (through 
programmatic threads) into grades K-12. Three of the themes are 
common to all of the documents: Models, Systems, and Patterns 
of Change. These three themes were included in the 1996 NAEP 
Science Assessment because they constitute major, interdiscipli- 
nary organizing principles of science. Eurther, they do not conflict 
or compete with the factual content of the various fields, but rather 
augment and help organize that information into a coherent intel- 
lectual framework. 

Models of objects and events in nature can be used to understand 
complex or abstract phenomena. Models may be first attempts to 
identify the relevant variables to build evermore useful representa- 
tions, or they may be highly refined for predictions about the actual 
phenomenon. Students need to understand the limitations and sim- 
plifying assumptions that underlie the many models used in the nat- 
ural sciences. A model is likely to fit data well only within a limited 
range of circumstances and to be misleading outside of that range. 

Systems are complete, predictable cycles, structures, or processes 
occurring in natural phenomena, but students should understand that 
the idea of a system is an artificial construction created by people 
for certain purposes, for example, to gain a better understanding of 
the natural world or to design an effective technology. The construct 
of a system entails identifying and defining its boundaries, identify- 
ing its component parts and the interrelations and interconnections 
among those parts, and identifying the inputs and outputs of the 
system. 

Regardless of the topic around which the Patterns of Change 
theme is developed, students should be able to recognize patterns 
of similarity and difference, to perceive how these patterns change 
over time, to remember common types of patterns, and to transfer 
their understanding of a familiar pattern of change to a new and un- 
familiar situation. Appendix C contains more detail on these three 
themes and the developmentally appropriate expectations for stu- 
dents at grades 4, 8, and 12. 
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Chapter Three 

Desired Attributes of the Assessment 



Assessment Issues 

N AEP functions to provide information about (1) the knowl- 
edge and scientific understanding of the nation’s youth and 
(2) the features of science education programs that relate to 
high levels of student achievement. Consequently, the design plan 
for the NAEP Science Assessment includes strategies for measure- 
ment in both of these areas. Research has produced practical and 
theoretical knowledge that is important to the assessment design 
process. 

The Nature of Testing and the Nature of Knowledge and 
Learning 

Except for early assessments, NAEP Science Assessments have 
consisted primarily of short-answer, paper-and-pencil questions that 
were mostly multiple choice. The previous assessments tended to 
focus on discrete components of science, each of which was usual- 
ly learned independently of the others. Hence, tests were made up 
of independent items, each comparable to the others and weighted 
the same. 

This Eramework is based on a different view. It holds that scien- 
tific knowledge should be organized to provide a structure that con- 
nects and creates meaning for factual information, and that this 
organization is influenced by the context in which the knowledge is 
presented. Eeaming is perceived as an activity in which the learner 
interacts with the physical world, with peers and teachers, and with 
the scientific community. In this view, science proficiency depends 
on the ability to know facts and integrate them into larger constructs 
and to use the tools, procedures, and reasoning processes of science 
for an increased understanding of the natural world. 

Rather than concentrate on facts in isolation, assessments will 
reflect the organization and structure of scientific knowledge and 
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the nature of learning in science. Because scientific knowledge is 
expanding faster than can he accommodated hy any curriculum, 
teachers and assessment designers must make choices about what 
topics, concepts, and factual information to address. Consequently, 
the NAEP Science Assessment Framework concentrates on assess- 
ing students’ ability to relate basic facts and concepts as well as 
their ability to discuss and evaluate approaches to science -related 
problems. The Framework also stresses that an assessment of what 
students know and can do must employ techniques reflective of the 
nature of science. 

Inferring Understanding From Student Responses 

Test items present students with tasks that may require a range 
of responses, from recall of factual information to performance of 
scientific investigations to complex reasoning. Based on analysis of 
the responses, experts in the field make inferences about students’ 
understanding; that is, their knowledge and reasoning skills that are 
assumed to have produced the responses. The validity of these in- 
ferences is a central issue in assessment. 

Responses to exercises designed to assess thinking or mental 
processing are generally more difficult to interpret than responses 
to items designed to assess factual knowledge. In practice, basing 
an assessment of the quality of mental processing on short respon- 
ses is problematic. Often the decision about the mental processes 
applied is based only on the accuracy of the factual knowledge in 
the answer. When the answer is factually correct, the observer in- 
fers that the mental processes represent scientific reasoning (for 
example, those mental processes that are necessary to understand 
information in the test question stem, retrieving scientific knowledge 
from memory, reasoning from the stem to the correct response, or 
eliminating incorrect responses). However, this is not necessarily a 
valid inference. An incorrect answer may be the result of misinfor- 
mation, not flawed reasoning; a correct answer is not necessarily 
the product of sound reasoning. Illogical thinking or using a wrong 
assumption or incorrect information can produce a seemingly cor- 
rect answer. Furthermore, a correct answer may not require any 
higher order thinking at all; it simply may have been recalled. Only 
if the student response includes some indication of how the answer 
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was obtained will those who score the assessment have information 
from which to choose alternative interpretations. 

Science entails observing objects and phenomena in the natural 
world and collecting and interpreting information about them. For 
this reason, pencil-and-paper tests have been criticized as too limit- 
ed to assess what students know and can do in science. Over the 
past several years, groups have developed assessment exercises that 
engage students in performance tasks using scientific equipment 
and materials. Student responses are recorded by an observer or by 
the students themselves in written form. However, these exercises 
have limitations as well. 

Indeed, in the course of developing the new Framework, numer- 
ous examples of performance exercises were examined with respect 
to science concepts and reasoning, but many did not stand up to 
rigorous analysis. These exercises might have led science teachers 
and educators to draw faulty inferences. 

The following is typical of a performance exercise that is coun- 
terproductive: an exercise requires a student to identify several un- 
known substances by means of indicators, but the student is given 
minutely detailed directions for performing each step in the identi- 
fication process. Unfortunately, even if the answers are correct, the 
only inference to be drawn is that the student can follow written in- 
structions. A test item formulated with such detailed, step-by-step 
directions reduces to zero the science understanding needed for 
problemsolving. 

Many of the so-called performance assessment tasks that were 
reviewed turned out to be standard laboratory exercises that, again, 
were reduced to “follow-the-instructions” problems. No inferences 
about a student’s knowledge of science or its tools and procedures 
can be drawn from such exercises. To test higher order thinking 
skills — a major goal of performance assessments — problems need 
to be placed in new contexts, applied to new situations, or have 
new elements introduced that preclude students’ recalling what 
they have done before (Resnick, 1987). 

Class, culture, ethnicity, gender, language ability, and access to 
quality instruction may influence the manner in which science is 
learned and the manner in which science attitudes and knowledge 
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are produced. Hence, individuals need opportunities to demonstrate 
knowledge or competencies in different contexts. 

Assessment techniques that show group differences are more 
likely to reveal problems with student learning and classroom in- 
struction than with assessment per se. However, this does not elim- 
inate the assessment community’s responsibility to the broader 
society. In addressing the issues of pluralism, multiple assessment 
methods may be more effective than any one method, no matter 
how well it is developed. 

Developments in Assessment 

Currently, some state pilot testing efforts are providing new ideas 
about assessment exercise and task formats. These pilot activities 
are also aimed at assessing new types of information relevant to 
new curriculum guides emerging in the states. 

The experimental assessment work (much of it pioneered in the 
United Kingdom) uses new approaches, including performance 
tasks, open-ended tasks, and new types of multiple-choice ques- 
tions that are thematic or conceptual or that ask students to explain 
their choices in short written answers. Moreover, state assessments 
are experimenting with collecting information on other meaningful 
outcomes: sustained student work, proficiency in designing and 
conducting experiments, and fluency of ideas critical to the natural 
sciences and related fields. They are also experimenting with inno- 
vative reporting approaches. The performance measures are created 
through consensual judgment about what students should know and 
be able to do at given grade levels or developmental stages. 

Criteria for Assessing Learning and Achievement 
in Science 

The Framework for the NAEP Science Assessments has been 
developed according to the following broad guidelines: 

• By focusing on meaningful knowledge and skills, NAEP 
should be a force in fostering progress as well as measuring 
it, enabling more students to learn more science. 
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• A range of assessment means must be used, including some 
in which the student is required to create and construct, not 
just to recognize and respond. 

• Assessment exercises should challenge students at develop- 
mentally appropriate levels to: 

— explain commonplace natural phenomena using appropriate 
scientific theory, principles, and concepts; 

— ^plan the investigation of a novel scientific problem; 

— demonstrate understanding of the basic knowledge struc- 
tures of science by using the appropriate techniques to 
connect concepts to one another and to the theory within 
which they are embedded; 

— demonstrate some understanding of pervasive crosscutting 
themes in science; 

— solve practical problems by using the appropriate theories, 
principles, concepts, and techniques of science. 

• Assessments must be sensitive to the need and ability of stu- 
dents to function in a variety of contexts. 

• Assessment exercises should use a variety of formats to allow 
students to display the wide range of competencies expected 
as outcomes of science education. 

• Assessment tasks that are larger than single items should be 
analyzed in multiple ways, not restricted to providing infor- 
mation for single scales. 

• Test results should not be normalized; that is, students’ out- 
comes should not be manipulated to fit a normal distribution 
curve. 

• Assessments must have enough questions about enough top- 
ics to explore students’ knowledge in depth. 

Achievement Levels in Science 

Achievement levels describe how well students perform on the 
content and thinking levels required by the assessment. They evaluate 
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the quality of the outcomes of students’ education in science at 
grades 4, 8, and 12 as measured hy NAEP. 

Three achievement levels — Basic, Proficient, and Advanced — 
have been defined for each grade level assessed hy NAGB : 

• Basic denotes partial mastery of prerequisite knowledge and 
skills that are fundamental for proficient work at each grade. 

• Proficient represents solid academic performance for each 
grade assessed. Students reaching this level have demonstrat- 
ed competency over challenging subject matter, including 
subject matter knowledge, application of such knowledge to 
real-world situations, and analytical skills appropriate to the 
subject matter. 

• Advanced represents superior performance. 

Appendix A lists the final achievement level descriptions for stu- 
dents participating in the 2005 NAEP Science Assessment in 
grades 4, 8, and 12. The assessment was constructed to measure 
and report student performance according to the three levels of 
achievement, as required by NAEP policy. 
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Chapter Four 

Characteristics of Assessment 
Exercises 



Types of Exercises 

I nnovative assessments in the United States and other countries 
use three major item types: performance exercises, open-ended 
paper- and-pencil exercises, and multiple-choice items probing 
understanding of conceptual and reasoning skills. A fourth type 
often added to time-limited tests is the collection and evaluation of 
portfolios of student work done in the course of instruction. There 
is also an emerging assessment technique that involves two-phase 
testing. The following sections discuss these exercise types and 
provide some guidelines for the amount of assessment time to he 
devoted to each. Further details are provided in the Specifications 
and Reporting Formats and Issues documents. 

Performance Exercises 

In performance exercises, students actually manipulate selected 
physical ohjects and try to solve a scientific problem about the 
objects before them. Although various types of performance tests 
have been piloted extensively, their standardization and administra- 
tion differ widely. One method for ensuring uniform administration 
is the use of standardized performance test kits, with each test 
proctored and scored by trained personnel. Depending on the ob- 
jectives established for the assessment, student answer sheets can 
also be used to provide responses to be scored. To adequately 
measure the goals outlined for the assessment, performance items 
generally should make up at least 30 percent of the assessment, as 
measured by student response time. An extra period of time (20 or 
30 minutes) may be necessary for students who have been assigned 
to perform complex tasks. 

The shortcomings of many performance tasks currently being used 
were discussed in the preceding chapter. How can a performance 
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exercise be designed so that it meets criteria for assessing science 
concepts and their relations? The exercise should be meaningful 
and not a context-free laboratory problem. Personal context, for ex- 
ample, is seen in the following problem: “You have just been given 
this new drink. It is claimed to be sugar- and calorie-free. What can 
you find out about the accuracy of these claims with these indica- 
tors?” Students can be given the names and procedures for the safe 
use of each indicator, but no information about their scientific use. 
The students would have to know, for example, that iodine solution 
is a test for starch and what the negative and positive reactions are. 
The students would also have to know that if fats, proteins, or car- 
bohydrates are present, they will yield calories. They would have to 
plan how to conduct an investigation of the unknown in such a way 
as not to waste the materials, to be able to repeat the investigation 
when they believe their procedure is faulty, and to have enough so- 
lution left to replicate the investigation for verification. They would 
also have to design their data collecting procedures. Finally, they 
would need to interpret and justify their findings. The questions 
asked of students as part of a performance exercise need to enable 
the students to display understanding and to justify interpretations. 
Such questions as “What substance is in the unknown?” or “How 
far did the dye diffuse?” do not elicit responses that demonstrate 
understanding. 

If students need additional information to carry out the task, 
they could be asked before they begin if they would like any other 
materials and why. If they request known substances for each indi- 
cator to refresh their memories, those could be provided. Such a 
request demonstrates one aspect of understanding the processes 
of science — knowing what one does not know and how to acquire 
more information. Scoring for such a problem could give points for 
science knowledge, for laboratory procedures, and (if using an ob- 
server) for a systematic approach to problemsolving in contrast to 
a trial-and-error or random approach. 

Open-Ended Paper-and-Pencil Items 

Open-ended items that require written responses provide partic- 
ularly useful insights to students’ levels of conceptual understanding. 
They can also be used to assess students’ abilities to communicate 
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in the sciences. In addition, open-ended items, if carefully crafted, 
can be used to reflect students’ abilities to generate rather than 
recognize information related to scientific concepts and their inter- 
connections. Open-ended items should make up at least 50 percent 
of the assessment, as measured by student response time. About 
one-third of the open-ended questions should consist of extended 
response items. 

Multiple-Choice Items 

The use of multiple-choice items should be considered carefully 
because they are often overused to test low-level recall. Balanced 
with other item types, however, multiple-choice items are worth- 
while for measuring knowledge of important facts and concepts as 
well as deductive reasoning skills. Multiple-choice items should 
make up no more than 50 percent of the assessment, as measured 
by student response time. 

Additional Considerations 

Performance exercises, open-ended paper-and-pencil items, and 
multiple-choice items could produce responses less subject to faulty 
interpretation if students are given an opportunity to explain their 
responses, their reasoning processes, or their approach to a problem. 
Hence, an assessment should afford this opportunity. However, care 
must be taken, particularly with fourth graders, that language abili- 
ty is not confounded with science ability. This caution also applies 
to the more complex multiple-choice items needed to probe con- 
ceptual understanding. 

The new NAEP Science Assessment, to be consonant with cur- 
rent reform efforts in science education, needs to probe students’ 
depth of knowledge and scientific understanding. For this reason, 
it is recommended that for at least half the students sampled, the 
assessment should include an indepth examination involving a sin- 
gle problem or topic. The format could be a set of linked perform- 
ance tasks, open-ended paper-and-pencil exercises, multiple-choice 
items, or a combination thereof. Pending modification after pilot 
testing, the suggested time to be spent by students on this type of 
exercise is 10 minutes for grade 4; 20 minutes for grade 8; and 30 
minutes for grade 12. 
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Pilot Testing 

Multiple approaches need to be tried in the pilot testing of the 
assessment exercises. It would be especially useful to test the same 
concept(s) and performance skill(s) in different ways to see which 
method provides the richest, most reliable, and valid information. 
For example, if an open-ended question can easily be turned into a 
multiple-choice question without losing its intent and validity, it 
should be multiple choice. Open-ended questions should tap skills 
and knowledge that are truly open, probing for the integrated appli- 
cation of relevant knowledge, not for the recall of a series of un- 
connected facts. The following additional issues need to be 
investigated during pilot testing: 

• Scoring rubrics. These should be developed a priori for open- 
ended questions and performance tasks, but modified on the 
basis of pilot test results. Weights should be assigned within 
scoring rubrics of complex items to reflect the quality of the 
responses. Distinct rubrics must be developed to score for 
multiple aspects or for items contributing to more than one 
component of the assessment. 

• Scaffolding. To what extent should scaffolding, or providing 
additional information, be integrated into open-ended ques- 
tions? This could be done either in the form of hints to be 
given only after students get stuck (for example, in a computer- 
administered or individually timed exercise) or by informing 
the students how their answers will be scored. How much 
should students be able to learn during the test, as contrasted 
to what they learned in the classroom? Unfortunately, many 
science items in current tests require no science knowledge at 
all because everything is provided in the question stem or in 
the instructions. On paper- and-pencil tests, these questions 
usually turn out to measure general reasoning or scale and 
graph reading. 

• Student self-evaluation. Should students (perhaps in grades 
8 and 12 only) be able to evaluate their own test performance? 
Students might be given credit for what they know they do 
not know, particularly if they can articulate what steps they 
might take to find out the answer. Alternatively, they might 
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be asked to indicate how well they think they did on some 
items (metacognitive questions). 



Procedures for the Development of the Assessment 

The planning committee responsible for developing the NAEP 
Science Assessment Framework was concerned that the nature and 
specifics of the Framework be faithfully mirrored in the actual in- 
strument of the assessment. The committee therefore recommended 
that a detailed review of individual items and of each proposed as- 
sessment as a whole be conducted at the conclusion of the follow- 
ing four stages: 

• item development and selection for pilot testing; 

• analysis of data from the pilot test and review of the results; 

• a priori development of the scales at each grade level, together 
with review of the items used for behavioral anchoring; 

• selection of items and formulation of the assessment as a 
whole for the NAEP test administration. 

The review committee(s) should be broadly constituted to in- 
clude scientists; teachers; science educators and researchers; cogni- 
tive psychologists; psychometricians; and informed members of the 
lay community, including representatives from business, higher ed- 
ucation, education governing bodies, and parent groups. The stages 
of review should be carried out by the same committee. If this is 
not feasible, the different committees need to have overlapping 
membership. Members of the planning committee should be in- 
cluded throughout the entire review process. 

Special Study in Science 

Special studies are often recommended as a part of the National 
Assessment process because new or emerging techniques offer 
promise and, if they yield useful information, will make a positive 
contribution to the assessment process. Special studies are a part of 
the main NAEP process and are usually reported along with the re- 
sults from the national sample. 

In 1996, a special study was carried out to assess students with 
advanced training in science. Past NAEP Science Assessments 
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have been criticized for having too low a ceiling; that is, for not in- 
cluding an adequate number of items at advanced levels of difficulty. 
As a result, NAEP assessments are assumed not to have reflected 
what the best prepared students know or can do in science. An 
international assessment conducted in 1995 included students who 
are concentrating on science in high school. It would be highly 
desirable to have similar results from NAEP. 

Suggestions on how to include the most competent students in 
this special study included sampling students who are taking ad- 
vanced courses in science. Assessments must contain a sufficient 
number of challenging exercises to measure what these “best” stu- 
dents know and are able to do. It was recommended, therefore, that 
a special study be conducted in 1996 with a subsample of the na- 
tional NAEP sample to determine whether this is a useful approach 
to establish the achievement and performance of the best science 
students. The results of this special study. Assessing the Best: 
NAEP’s 1996 Assessment of Twelfth-Graders Taking Advanced Sci- 
ence Courses can be found on the NAEP website at www.nces.ed. 
gov/nationsreportcard. 
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Appendix A 

NAEP Science Achievement Level Descriptions 




NAEP Science Achievement Level 
Descriptions 



Grade 4 
Basic 

Students performing at the Basic level demonstrate some of the 
knowledge and reasoning required for understanding the Earth, phys- 
ical, and life sciences at a level appropriate to grade 4. For example, 
they can carry out simple investigations and read uncomplicated 
graphs and diagrams. Students at this level also show a beginning 
understanding of classification, simple relationships, and energy. 

Fourth-grade students performing at the Basic level are able to 
follow simple procedures, manipulate simple materials, make obser- 
vations, and record data. They are able to read simple graphs and 
diagrams and draw reasonable but limited conclusions based on data 
provided to them. These students can recognize appropriate experi- 
mental designs, although they are unable to justify their decisions. 

When presented with diagrams, students at this level can identi- 
fy seasons; distinguish between day and night; and place the posi- 
tion of the Earth, Sun, and planets. They are able to recognize 
major energy sources and simple energy changes. In addition, they 
show an understanding of the relationship between sound and vi- 
brations. These students are able to identify organisms by physical 
characteristics and group organisms with similar physical features. 
They can also describe simple relationships among structure, func- 
tion, habitat, life cycles, and different organisms. 

Proficient 

Students performing at the Proficient level demonstrate the knowl- 
edge and reasoning required for understanding the Earth, physical, 
and life sciences at a level appropriate to grade 4. For example, they 
understand concepts relating to the Earth’s features, physical prop- 
erties, structure, and function. In addition, students can formulate 
solutions to familiar problems as well as show a beginning aware- 
ness of issues associated with technology. 
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Fourth-grade students performing at the Proficient level are able 
to provide an explanation of day and night when given a diagram. 

They can recognize major features of the Earth’s surface and the 
impact of natural forces. They are also able to recognize water in 
its various forms in the water cycle and can suggest ways to con- 
serve it. These students recognize that various materials possess 
different properties that make them useful. Students at this level 
are able to explain how structure and function help living things 
survive. They have a beginning awareness of the benefits and chal- 
lenges associated with technology and recognize some human ef- 
fects on the environment. They can also make straightforward 
predictions and justify their position. 

Advanced 

Students performing at the Advanced level demonstrate a solid 
understanding of the Earth, physical, and life sciences as well as 
the ability to apply their understanding to practical situations at a 
level appropriate to grade 4. Eor example, they can perform and 
critique simple investigations, make connections from one or more 
of the sciences to predict or conclude, and apply fundamental con- 
cepts to practical applications. 

Eourth-grade students performing at the Advanced level are able 
to combine information, data, and knowledge from one or more of 
the sciences to reach a conclusion or to make a valid prediction. 
They can also recognize, design, and explain simple experimental 
procedures. 

Students at this level recognize nonrenewable sources of energy. 
They also recognize that light and sound travel at different speeds. 
These students understand some principles of ecology and are able 
to compare and contrast life cycles of various common organisms. 
In addition, they have a developmental awareness of the benefits 
and challenges associated with technology. 
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Grade 8 



Basic 

Students performing at the Basic level demonstrate some of the 
knowledge and reasoning required for understanding the Earth, phys- 
ical, and life sciences at a level appropriate to grade 8. For exam- 
ple, they can carry out investigations and obtain information from 
graphs, diagrams, and tables. In addition, they demonstrate some 
understanding of concepts relating to the solar system and relative 
motion. Students at this level also have a beginning understanding 
of cause-and-effect relationships. 

Eighth-grade students performing at the Basic level are able to 
observe, measure, collect, record, and compute data from investiga- 
tions. They can read simple graphs and tables and are able to make 
simple data comparisons. These students are able to follow direc- 
tions and use basic science equipment to perform simple experi- 
ments. In addition, they have an emerging ability to design 
experiments. 

Students at this level have some awareness of causal relation- 
ships. They recognize the position of planets and their movement 
around the Sun and know basic weather-related phenomena. These 
students can explain changes in position and motion such as the 
movement of a truck in relation to that of a car. They also have an 
emerging understanding of the interrelationships among plants, ani- 
mals, and the environment. 

Proficient 

Students performing at the Proficient level demonstrate much 
of the knowledge and many of the reasoning abilities essential for 
understanding the Earth, physical, and life sciences at a level ap- 
propriate to grade 8. For example, students can interpret graphic 
information, design simple investigations, and explain such scien- 
tific concepts as energy transfer. Students at this level also show an 
awareness of environmental issues, especially those addressing en- 
ergy and pollution. 

Eighth-grade students performing at the Proficient level are able 
to create, interpret, and make predictions from charts, diagrams. 
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and graphs based on information provided to them or from their 
own investigations. They have the ability to design an experiment 
and have an emerging understanding of variables and controls. 
These students are able to read and interpret geographic and topo- 
graphic maps. In addition, they have an emerging ability to use and 
understand models, can partially formulate explanations of their 
understanding of scientific phenomena, and can design plans to 
solve problems. 

Students at this level can begin to identify forms of energy and 
describe the role of energy transformations in living and nonliving 
systems. They have knowledge of organization, gravity, and motion 
within the solar system and can identify some factors that shape 
the surface of the Earth. These students have some understanding 
of properties of materials and have an emerging understanding of 
the particulate nature of matter, especially the effect of temperature 
on states of matter. They also know that light and sound travel at 
different speeds and can apply their knowledge of force, speed, and 
motion. These students demonstrate a developmental understanding 
of the flow of energy from the Sun through living systems, espe- 
cially plants. They know that organisms reproduce and that charac- 
teristics are inherited from previous generations. These students 
also understand that organisms are made up of cells and that cells 
have subcomponents with different functions. In addition, they are 
able to develop their own classification system based on physical 
characteristics. These students can list some effects of air and water 
pollution as well as demonstrate knowledge of the advantages and 
disadvantages of different energy sources in terms of how they af- 
fect the environment and the economy. 

Advanced 

Students performing at the Advanced level demonstrate a solid 
understanding of the Earth, physical, and life sciences as well as 
the abilities required to apply their understanding in practical situa- 
tions at a level appropriate to grade 8. Eor example, students can 
perform and critique the design of investigations, relate scientific 
concepts to each other, explain their reasoning, and discuss the im- 
pact of human activities on the environment. 
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Eighth-grade students performing at the Advanced level are able 
to provide an explanation for scientific results. They have a modest 
understanding of scale and are able to design a controlled experi- 
ment. These students have an understanding of models as represen- 
tations of natural systems and can describe energy transfer in living 
and nonliving systems. 

Students at this level are able to understand that present physical 
clues, including fossils and geological formations, are indications 
that the Earth has not always been the same and that the present is 
a key to understanding the past. They have a solid knowledge of 
forces and motions within the solar system and an emerging under- 
standing of atmospheric pressure. These students can recognize a 
wide range of physical and chemical properties of matter and some 
of their interactions and understand some of the properties of light 
and sound. Also, they can infer relationships between structure and 
function. These students know the differences between plant and 
animal cells and can apply their knowledge of food as a source of 
energy to a practical situation. In addition, they are able to explain 
the impact of human activities on the environment and the economy. 

Grade 12 
Basic 

Students performing at the Basic level demonstrate some knowl- 
edge and certain reasoning abilities required for understanding the 
Earth, physical, and life sciences at a level appropriate to grade 12. 
In addition, they demonstrate knowledge of the themes of science 
(models, systems, and patterns of change) required for understand- 
ing the most basic relationships among the Earth, physical, and life 
sciences. They are able to conduct investigations, critique the de- 
sign of investigations, and demonstrate a rudimentary understand- 
ing of scientific principles. 

Twelfth-grade students performing at the Basic level are able to 
select and use appropriate simple laboratory equipment and write 
down simple procedures that others can follow. They also have a 
developmental ability to design complex experiments. These stu- 
dents are able to make classifications based on definitions such as 
physical properties and characteristics. 
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Students at this level demonstrate a rudimentary understanding 
of basic models and can identify some parts of physical and bio- 
logical systems. They are also able to identify some patterns in na- 
ture and rates of change over time. These students have the ability 
to identify basic scientific facts and terminology and have a rudi- 
mentary understanding of the scientific principles underlying such 
phenomena as volcanic activity, disease transmission, and energy 
transformation. In addition, they have some familiarity with the ap- 
plication of technology. 

Proficient 

Students performing at the Proficient level demonstrate the 
knowledge and reasoning abilities required for understanding the 
Earth, physical, and fife sciences at a level appropriate to grade 12. 
In addition, they demonstrate knowledge of the themes of science 
(models, systems, and patterns of change) required for understand- 
ing how these themes illustrate essential relationships among the 
Earth, physical, and fife sciences. They are able to analyze data and 
apply scientific principles to everyday situations. 

Twelfth-grade students performing at the Proficient level are 
able to demonstrate a working ability to design and conduct scien- 
tific investigations. They are able to analyze data in various forms 
and utilize information to provide explanations and to draw reason- 
able conclusions. 

Students at this level have a developmental understanding of 
both physical and conceptual models and are able to compare vari- 
ous models. They recognize some inputs and outputs, causes and 
effects, and interactions of a system. In addition, they can correlate 
structure to function for the parts of a system that they can identify. 
These students also recognize that rate of change depends on initial 
conditions and other factors. They are able to apply scientific con- 
cepts and principles to practical applications and solutions for 
problems in the real world and show a developmental understand- 
ing of technology, its uses, and its applications. 

Advanced 

Students performing at the Advanced level demonstrate the knowl- 
edge and reasoning abilities required for a solid understanding of 
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the Earth, physical, and life sciences at a level appropriate to grade 
12. In addition, they demonstrate knowledge of the themes of sci- 
ence (models, systems, and patterns of change) required for inte- 
grating knowledge of scientific principles from the Earth, physical, 
and life sciences. Students can design investigations that answer 
questions about real-world situations and use their reasoning abili- 
ties to make predictions. 

Twelfth-grade students performing at the Advanced level are 
able to design scientific investigations to solve complex, real-world 
situations. They can integrate, interpolate, and extrapolate informa- 
tion embedded in data to draw well-formulated explanations and 
conclusions. They are also able to use complex reasoning skills to 
apply scientific knowledge to make predictions based on condi- 
tions, variables, and interactions. 

Students at this level recognize the inherent strengths and limita- 
tions of models and can revise models based on additional informa- 
tion. They are able to recognize cause- and-effect relationships 
within systems and can utilize this knowledge to make reasonable 
predictions of future events. These students are able to recognize 
that patterns can be constant, exponential, or irregular and can 
apply this recognition to make predictions. They can also design a 
technological solution for a given problem. 
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Appendix B 

Fields of Science 




Fields of Science 



The details on the fields of science provided helow indicate 
themes that can he explored within major topic areas as well as 
brief examples of developmentally appropriate sample questions 
that can form the basis for assessment exercises. 

Earth Science 

Earth science is the study of the planet Earth’s composition, 
processes, environments, and history, focusing on the solid Earth 
(lithosphere) and its interactions with air (atmosphere) and water 
(hydrosphere). In Earth science, the content to be assessed centers 
on objects (including bodies and materials and their composition, 
features, and structures) as well as processes and events that are 
relatively accessible or visible. The content includes objects such 
as soil, minerals, rocks and rock outcrops, fossils, rain, clouds, the 
Sun, and the Moon; processes such as erosion, deposition, weather, 
and climate; and events such as volcanic eruptions, earthquakes, 
and storms. 

The Solar System 
Earth/Moon/Sun : 

• observable evidence (Patterns of Change) 

• description and models (Systems, Models) 

Students should understand that the apparent daily motions of 
the Sun, Moon, planets, and stars are due to the rotation of the 
Earth about its axis every 24 hours. This rotation produces the 
Earth’s night-and-day cycle. They should also understand that the 
Earth’s 1-year revolution around the Sun changes how sunlight 
falls on one part or another of the Earth because of the tilt of the 
Earth’s axis, thus producing seasonal variations in climate. They 
should know that the combination of the Earth’s motion and the 
Moon’s own orbit around the Earth, once in about 28 days, results 
in the phases of the Moon. 
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The Earth and the Forces That Shape It 

With respect to the Earth, the NAEP Science Assessment should 
center on the following concepts: 

Climate (Patterns of Change): 

• water cycle and ground water (Systems) 

• pollution capacity (Systems, Models) 

• interior effects (Models, Systems, Patterns of Change) 

— crustal plates 

— rock cycles and strata 

Exterior Effects (Systems, Patterns of Change): 

• weathering 

• plants, animals, and civilization 

Students should understand that the Earth is a unique member of 
this solar system hut could he replicated in other galaxies in the 
universe; that it is at least 4.5 hillion years old and is a complex 
planet with several interacting systems; and that these systems have 
evolved through time and changes in them occur over periods that 
range from microseconds to millions of years and vary from sub- 
atomic to astronomical. Students should also understand that the 
Earth’s systems contain a variety of renewable and nonrenewable 
resources that sustain life (American Geological Institute, 1991). 

Climate: Atmosphere and Hydrosphere 

The atmosphere is the gaseous envelope that surrounds the 
Earth. It is continuously in motion, circulating in complex but reg- 
ular patterns and driven by direct and stored solar energy. There are 
strong interactions between the atmosphere and the hydrosphere 
that determine weather and climate, profoundly influencing human 
and all other life. Desired learning goals follow. 
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Grade 4: 

• Students are able to communicate what is special about air. 
What do their senses tell them about the air? What needs air? 

• Students are able to offer simple explanations for how the 
weather changes. How do people know when weather 
changes? How can changing weather conditions be measured? 

Grade 8: 

• Students know about basic weather-related phenomena 
(for example, tornadoes, hurricanes, drought, and acid 
precipitation). 

• Students know that relatively small changes in global tem- 
peratures can have dramatic effects on the Earth. 

• Students can access climatological information bases via 
computers and use other means to extract useful information. 

• Students can read a weather chart and extract basic 
information. 

• Students understand and can use simple instruments, such as 
barometers, that measure basic phenomena related to weather 
Students will make a barometer and/or know how it works. 

Grade 12: 

• Generally, 12th-grade students are able to connect relation- 
ships between atmospheric phenomena and long-term effects. 

• They understand that much of what determines the details of 
the weather depends on phenomena such as sea breezes, thun- 
derstorms, tornadoes, and wind shear. 

• They understand how scientists can monitor atmospheric 
events over time (for example, how products of pollution 
and carbon dioxide affect the poles). 

• Students are able to discuss and relate causes and possible 
solutions to their consequences and tradeoffs. 
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Water Cycle and Ground Water 

Desired learning goals follow. 

Grade 4: 

• Students understand that water exists not only on the surface 
of the Earth, hut beneath it as well. 

• Students understand that water is able to change the shape 
of the Earth. They can demonstrate their knowledge in two 
ways: by interpreting common local land features such as 
rivers, streams, mountain slopes, and delta deposits, or by de- 
signing simple models that illustrate their interpretations. 

• Students understand that most of the Earth’s surface is cov- 
ered by water. 

• Students know how waters of the Earth circulate. They can 
answer questions concerning where water is found; how water 
enters and leaves the atmosphere; and how water can be used 
more wisely. 

Grade 8: 

• Students know and demonstrate why water is special. What 
properties make water special? Where is water found: in the 
air, on Earth, and under the ground? 

• Students know that the oceans provide habitats for a wide 
variety of plant life and animal life. 

• Students can discuss some common problems that concern 
water, for example, its availability in their areas, shortages, 
relationship to supply and demand, and the effects of over- 
population on the availability and quality of potable water. 

• Students are able to understand common interactive cycles 
such as the water cycle, nitrogen cycle, and carbon cycle. 

• Students are familiar with some of the ways scientists explore 
the water environment. 
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Grade 12: 



• At grade 12, students are able to explain how water (or the 
lack of it) relates to their immediate state, city, or area. 

• They can discuss national issues related to water, such as acid 
precipitation and the effects of global warming. 

• They are able to understand how common cycles affect the 
climate of the area where they live. 

• Students know how to find answers to problems relating to 
how laws affect the use of water, as well as what hazards are 
associated with water and how they can be mitigated. 

Interaction of Earth’s Systems 

Desired learning goals follow. 

Grade 4: 

• Students understand the ways in which the Earth relates to the 
Sun (for example, periodicity, seasons, and the night-and-day 
cycle). 

• Students know about tides with respect to the Earth, Moon, 
and Sun. 

• Students understand basic facts about volcanoes and glaciers. 

• Students understand how rocks and minerals can be investi- 
gated, what they are made of, and how they form. 

Grade 8: 

• Students are able to understand how earthquake occurrences 
are recorded and note some positional regularities. 

• Students are able to connect short-term changes in climate 
with volcanic activity. 

• Students can discuss changes that have occurred in water lev- 
els, global temperature, and climate zones over the eons. 

They can advance some hypotheses as to why these changes 
have occurred. 
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• Students can identify how and where energy is obtained from 
the Earth; Earth materials that people use and where to find 
them; and the advantages and disadvantages of using the 
Earth’s resources. 

Grade 12: 

• Students in grade 12 can connect “the zone of fire” with plate 
tectonics. They understand how sliding plates cause sudden 
Earth movements. 

• They can discuss issues related to Earth systems (for exam- 
ple, why continents move, that they have not always been 
arranged the way they are now, and how human activity af- 
fects Earth systems). 

• Students can access data from more than one data source to 
develop an environmental impact statement for their 
region/town/block. 

• Students can discuss problems associated with agriculture and 
the lithosphere and relate these problems to changing atmos- 
pheric conditions. 

Physical Science 

The physical science component of the 2005 NAEP Science As- 
sessment should probe the following major topics: matter and its 
transformations, energy and its transformations, and the motion of 
things. Each topic is detailed below, together with some examples 
of topics appropriate for assessment at the different grade levels. 

Matter and Its Transformations 

With respect to matter and its transformations, the 2005 NAEP 
Science Assessment should center on the following concepts: 

“Many From Few:” 

• diversity of materials (Models) 

• classifications and types of materials (Patterns of Change) 

• particulate nature of matter (Models) 
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Temperature and States of Matter: 

• uses of materials 

— ^properties and uses 

— modifying properties by mixing, processing, and reacting 
(Patterns of Change, Models) 

— synthesis of materials with new properties (Patterns of 
Change, Models) 

Resource Management: 

• resource depletion and substitute materials (Models) 

• disposal and recycling (Systems) 

Materials encountered in the physical world differ greatly in 
shape, density, flexibility, texture, toughness, and color; in their 
ability to give off, absorb, bend, or reflect light; in the form they 
take at different temperatures; and in many other ways. Students 
need to understand that these varied substances are made up of rel- 
atively few kinds of basic materials (the atomic elements) com- 
bined in various ways. Only a few of these elements are abundant 
in the universe. 

As they advance in science, students should come to understand 
that the basic premise of the modem theory of matter is that mate- 
rials consist of a limited number of different kinds of atoms (ele- 
ments) that join together in different configurations to form 
substances. Thus, when substances react to form new substances, 
the elements composing them combine in new ways and the prop- 
erties of the substances created by the new combinations may be 
very different from those of the old. Almost every substance can 
exist in a variety of states (solid, liquid, and gaseous) depending on 
temperature and pressure. 

Patterns within the structure of matter have been elegantly des- 
cribed in the periodic table, an outstanding example of a model that 
provides a systematic view of matter and its interactions. Changes 
in temperature and changes in state represent a category of physical 
change among substances within the realm of matter and energy. 
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An understanding of the particulate nature of matter can be as- 
sessed through the identification of types of materials; for example, 
“mystery powders” or the equivalent at the elementary level. The 
effects of temperature and heat energy on systems are exemplified 
in how refrigerators work, how and why ice cubes melt, and how 
metallic fuses work. Some practical areas of human activity (for 
example, cooking and much of the modem chemical industry) and 
processes (transport of materials in biological systems) are related 
both to the nature of matter and to the effects of external factors on 
its behavior. All these topics can provide rich assessment exercises. 

The properties of matter determine the uses to which particular 
materials are put by manufacturers, engineers, and others involved 
in technology. Materials can be physically combined or processed 
to serve human needs. Modem materials technology has focused 
increasingly on the synthesis of materials with entirely new proper- 
ties. Chemical changes are typically involved, and the properties of 
the new materials (such as plastics and ceramics) may be entirely 
different from those of their components. 

The growth of technology has led to the use of some materials 
from the environment (for example, forests, ore deposits, and 
petroleum) much more rapidly than they can be replaced by natural 
processes. There is a continuing search for substitute materials, 
and, in many cases, they have been found or invented. 

Disposal of used materials has become an increasing problem. 
Some used materials, such as food scraps and waste paper, can be 
returned safely to the environment; however, as the population 
grows, that task becomes more difficult and expensive. Other mate- 
rials, such as aluminum scrap and glass, can be recycled, with re- 
sulting savings in energy and resources. Some materials, such as 
plastics, are not easily recycled, nor do they degrade quickly when 
returned to the environment. Other used materials — radioactive 
waste being the most dramatic but not the only example — are so 
hazardous for such a long time that it is not clear how best to dis- 
pose of them. This issue has become the subject of widespread de- 
bate and controversy. Solving these problems of disposal will 
require systematic efforts that include both social and technological 
innovations. Assessment questions dealing with the scientific and 
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technological issues involved in resource management are appro- 
priate for grades 8 and 12. 

Energy and Its Transformations 

With respect to energy and its transformations, the 2005 NAEP 
Science Assessment should center on the following concepts: 

Forms of Energy: 

• energy transformation (qualitative) and audits (quantitative): 
— living systems 

■ plants 

■ animals 

■ protista 

— natural physical systems 
— artificial (human-constructed) systems 

Energy Sources and Use: 

• quantity and kind 

• distribution (Patterns of Change) 

• energy conversions, heat gain/loss, and efficiency (Patterns 
of Change, Models) 

• slowing depletion of energy sources, or conservation 

• costs, implications, advantages, risks, and availability 
( Patterns of Change ) 

Students should understand that the concept of energy is central 
to understanding changes observed in natural and artificial systems. 
Observable changes occur when energy is added to a system, when 
energy is removed from a system, or when energy is transformed 
from one form to another. Energy appears in many forms and is 
categorized in different ways: light, heat, sound, kinetic and poten- 
tial (electromagnetic, electrical, chemical, gravitational, and elas- 
tic), consumable and renewable, and available and unavailable. 
Although various forms of energy appear to be very different, each 
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can be measured in a way that makes it possible to keep track of 
how much of one form is transformed into another. 

Students need to understand that energy is conserved. Within a 
system, whenever the quantity of energy in one place or form 
changes, the quantity of energy in another place or form increases 
or decreases by a similar quantity, and the total energy remains the 
same. Thus, if no energy leaks in or out across the boundaries of 
a system, the total energy of all the different forms in the system 
will not change, no matter what kinds of changes occur within the 
system. 

Energy transformations usually result in producing some ther- 
mal energy (heat), which leaks away by radiation or conduction (for 
instance, from engines, electrical wires, hot water tanks, human 
bodies, or stereo systems) and becomes unavailable for further 
transformations. Thus, the total quantity of energy available for 
transformation usually decreases. 

Students need to develop an understanding of the more general 
principle that natural processes occur in the direction of increasing 
the total disorder of the system and its surroundings. Although some 
subsystems do increase in orderliness (such as the freezing of water 
to form ice), at the same time, another part of the system or a con- 
nected system is becoming more disordered. The cells of a human 
organism, for example, are always busy increasing order by build- 
ing complex molecules and body structure. This occurs at the cost 
of increasing disorder even more through processes such as break- 
ing down the molecular structure and order of food that is eaten 
and warming up surroundings. 

Energy transformations occur both naturally and in devices 
constructed by humans. 

Naturally Occurring Transformations: 

• solar energy into stored energy such as starches, fats, and 
proteins 

• solar energy into heat 
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• potential energy into kinetic energy (the potential energy of 
roller coasters at the top of a hill converting to kinetic energy 
on the way to the hottom) 

Transformations Occurring in Human Artifacts: 

• electric mixer converts electrical energy into mechanical 
energy 

• hair dryer converts electrical energy into heat energy 

• automohile converts chemical energy into mechanical energy 

In the operation of these devices, as in all phenomena, the useful 
energy output (that is, what is available for further change) is al- 
ways less than the energy input, with the difference usually appear- 
ing as heat. One goal in the design of such devices is to make them 
as efficient as possible, that is, to maximize the useful output for a 
given input and to minimize wasted heat energy. 

Radiant energy from the Sun is the ultimate source of most of the 
energy we use. It becomes available to us in several ways. The ener- 
gy of sunlight is captured directly in plants, which then may be 
eaten; it also heats the air, land, and water, causing wind and rain. 
For much of history, burning wood was the most common source 
of intense energy for cooking, heating dwellings, and running ma- 
chines. Most of the energy used today is derived from burning fossil 
fuels, which contain stored solar energy that plants collected over 
millions of years. A new source of energy is the fission of the nuclei 
of heavy elements, which releases an immense quantity of energy in 
relation to the mass of material used when compared with the burn- 
ing of fossil fuels. In nuclear reactors, the energy generated is used 
mostly to heat water into steam, which drives electric generators. 

Humans use energy for technological processes such as trans- 
porting, manufacturing, communicating, and getting raw materials, 
then working with them, and finally recycling them. Students need 
to appreciate that different sources of energy and ways of using them 
involve different costs, implications, and risks. Some resources 
will continue to be available indefinitely; others can be made self- 
renewing, but only at a limited rate. Fuels like coal, oil, natural gas, 
and uranium will become more difficult to obtain as the most read- 
ily available sources become depleted. New technology may make 
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it possible to use the remaining sources better; the ultimate limita- 
tion may be prohibitive cost rather than complete disappearance. 

Students should know that the depletion of nonrenewable energy 
sources can be slowed by both technical and social means. Techni- 
cal means include maximizing the advantages realized from a 
given input of energy through good design of the transformation 
device, insulation to restrict heat flow (insulating hot water tanks), 
or additional work with the heat as it dissipates. Social means in- 
clude government, which may restrict low-priority uses of energy 
or establish requirements for efficiency (as in automobile engines) 
or insulation (as in house construction). Individuals may also 
make energy efficiency a consideration in their own choice and 
use of technology (for example, turning out lights and driving high- 
efficiency cars), either to conserve energy as a matter of principle 
or to reduce their personal long-term expenses. Students need to 
appreciate that there will always be tradeoffs. For example, better 
insulated houses restrict ventilation and thus may increase the in- 
door accumulation of pollutants. 

The bases for these energy-related concepts should be laid in el- 
ementary school science. The following examples illustrate appro- 
priate activities that can be used to formulate assessment exercises 
for students in grade 4: 

• The student gives examples from his/her own experience of 
heat energy and light energy changing a system. 

• The student identifies the source of energy for a familiar sys- 
tem (animal, plant, car, electric appliance) and describes some 
of the energy conversions that take place in each system. 

• The student identifies the energy stored in a stretched rubber 
band and in a stretched or compressed spring as potential en- 
ergy. The student explains that to store potential energy in a 
rubber band or in a spring, he/she must exert a force to stretch 
the rubber band, or to stretch or compress the spring. 

• Given pictures of several situations, some of which depict a 
force being exerted or work being done and some of which 
do not, the student identifies those pictures in which a force 
is being exerted. 
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• The student explains, using the words fuel and energy in con- 
text, why a candle goes out when the wax is used up. 

• The student writes a short essay on how his/her life would he 
different if all the coal and petroleum on Earth were used up. 

The Motion of Things 

With respect to this topic area, the 2005 NAEP Science Assess- 
ment Framework should center on the following concepts: 

Motion: 

• force and changes in position and motion 

• action and reaction 

Waves: 

• vibrations and waves as motion summaries 

• general wave behavior 

• electromagnetic radiation 

• effects of wavelength 

• interactions of electromagnetic radiation with matter 

What students at the three grade levels should understand about 
these concepts is summarized below. 

Reference Frames 

Grade 4 

Everything moves: bicycles, cars, and trains; the stars, planets, 
and moons; the Earth, its surface, and everything on its surface; 
and all living things and every part of all living things. Positions of 
things may be described, but positions may change. Monitoring 
changes in time yields information about speed. 

Grade 8 

No special point in space can serve as a reference for all other 
motion. All motion is relative to whatever point or object is chosen. 
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Forces and Motion 

Grade 4 

Changes in motion (that is, changes in speed or direction) are 
due to the effects of forces. 

Grade 8 

Any object maintains a constant speed and direction of motion 
(including being at rest) unless an unbalanced outside force acts on 
it. When an unbalanced force does act on an object, the object’s 
motion changes. Depending on the direction of the force relative to 
the direction of motion, the object may change its speed (a falling 
apple), its direction of motion (the Moon in its curved orbit), or 
both (a fly ball). The greater the extent of the unbalanced force, the 
more rapidly a given object’s speed or direction of motion changes. 
In most familiar situations, friction between surfaces brings forces 
into play that complicate the description of motion, although the 
basic principles still apply. 

Grade 12 

The more massive an object is, the less rapidly its speed or di- 
rection changes in response to any given force. Whenever A exerts 
a force on B, B exerts an equal force back on A, but in the direc- 
tion opposite of the force exerted by A. 

Vibrations and Waves 

Grade 4 

Some motions can be described most conveniently in summary 
descriptions of the pattern of motion, such as vibrations and waves. 
Vibrations may set up a traveling disturbance that spreads away 
from its source. 

Grade 8 

Vibration involves parts of a system moving back and forth in 
much the same place, so the motion can be summarized by how 
frequently it is repeated and by how far the parts of a system are 
displaced during the cycle. Vibration may move through a system 
as a wave. Wave behavior can be described in terms of speed. 
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wavelength, and frequency. Wavelength can help determine how a 
wave interacts with things: how well it is transmitted, ahsorhed, re- 
flected, or diffracted. 

Grade 12 

Apparent change in wavelength can provide information about 
relative motion. The ways in which shock waves of different wave- 
lengths travel through and reflect from layers of rock are important 
clues to the structure of the Earth’s interior. 

Light as Waves 

Grade 4 

White light is made up of all different colors of light. Things ap- 
pear to have different colors because they reflect or scatter the light 
of some colors more than others. 

Grade 8 

Light behaves in many ways like waves: changing direction, 
bouncing off surfaces, spreading out, speeding up, slowing down, 
and changing wavelength. 

Grade 12 

The interaction of electromagnetic waves with matter varies 
greatly with wavelength. Thus, different but somewhat overlapping 
electromagnetic ranges have been given distinctive names: radio 
waves, microwaves, radiant heat or infrared radiation, visible light, 
ultraviolet light, x-rays, and gamma rays. Materials that allow one 
range of wavelengths to pass through them may completely absorb 
others. For example, some gases in the atmosphere, including car- 
bon dioxide and water vapor, are transparent to much of the incom- 
ing sunlight but not to the infrared radiation emitted by the warmed 
surface of the Earth. Consequently, heat energy is trapped in the at- 
mosphere. The temperature of the Earth rises until its total radia- 
tion output reaches a state of balance with the total radiation input 
from the Sun. 
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Life Science 

The fundamental goal of life science is to attempt to understand 
and explain the nature of life. During the 20th century, the thrust of 
biological research changed its focus from descriptive natural his- 
tory to experimental science, with most biological investigations 
conducted within the theory of evolution. The major concepts to be 
assessed in the life sciences (with evolution as the central, unifying 
theory) are listed below and developed further in the grade-level 
descriptions that follow. 

Cells: 

• information transfer 

• energy transfer 

• cellular communication 

Organisms: 

• reproduction, growth, and development 

• life cycles 

• functions and interactions of systems within organisms 

Ecology: 

• the interdependence of life: populations, communities, and 
ecosystems 

Evolution: 

• the diversity of life on Earth 

• genetic variation within a species 

• adaptation and natural selection 

• changes in diversity over time 

The three themes in the NAEP Science Assessment Framework 
(Patterns of Change, Systems, and Models) can be interwoven with 
these major concepts in the life sciences. Because evolution is the 
major pattern of change that occurs in the life sciences, the Patterns 
of Change theme can enhance understanding of all the life science 
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concepts listed above. Because the Systems theme can pertain to sys- 
tems at the cellular, organismal, population, community, and ecosys- 
tem levels, this theme can also enhance understanding of most, if not 
all, life science concepts. Although Models are used in the life sci- 
ences, this theme receives less emphasis in life science than in the 
physical and Earth sciences, particularly at grades 4 and 8. 

Students’ understanding of life science concepts develops gradu- 
ally as they proceed from grade 4 to grade 8 to grade 12. A de- 
scription of the developmentally appropriate concepts that should 
be understood at each grade follows. 

Grade 4 

Organisms: 

• As some animals grow, they look pretty much the same, they 
just increase in size. As other animals grow, they change from 
one form to another form that looks very different. They may 
change form several times before they become adults. 

• Only adults can reproduce, but not all young animals survive 
long enough to become adults. 

• Many activities go on inside the body that cannot be seen. 
When something happens in one part of the body, it affects 
what goes on in other parts of the body. 

Ecology: 

• Plants make their own food with sunlight, water, and air. 

• Some animals eat plants; some of these animals are in turn 
eaten by other animals. 

• Plants and animals get energy and building materials from 
their food. 

Evolution: 

• There are different kinds of plants and animals on Earth and 
in the sea. 

• There are differences among individuals of the same kind of 
plant or animal. 
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• Children of the same parents are somewhat alike and some- 
what different. 

Grade 8 

Organisms: 

• Different systems of the body have different functions; how- 
ever, the functioning of each system affects other systems. 

• Interactions among systems are complex. These interactions 
maintain a fairly stable operation of the entire system that can 
resist disturbance from within or without. 

• Interaction with other organisms (especially microorganisms) 
is important to maintain health or cause disease. Avoiding or 
killing harmful microorganisms can prevent disease. 

Ecology: 

• Plants use energy in sunlight to assemble food molecules 
from water and carbon dioxide. 

• Plants and animals break down food molecules to obtain food 
energy. 

• The source of energy and materials for all animals is plants. 

• The pattern of “what eats what” in a community can be 
complex. 

Evolution: 

• The organisms that survive long enough to reproduce may be 
different in some ways from others in a population that do not 
survive long enough to reproduce; their offspring may inherit 
the anatomical, chemical, and/or behavioral characteristics 
that enabled the parents to survive. 

• Gradually, over many generations, organisms with favorable 
characteristics may crowd out other organisms in the popula- 
tion that do not have these characteristics. 

• Scientists believe that these processes, operating over very 
long periods of time, have resulted in the diversity of organ- 
isms that can be seen on Earth today. 
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• Adaptation may be to either the living or nonliving compo- 
nents of the environment. 

Grade 12 

Cells: 

• Every cell contains a recipe for running the cell, coded in de- 
oxyribonucleic acid (DNA) molecules; the code mainly speci- 
fies how to put proteins together. 

• During cell reproduction, the information in the DNA code is 
passed on to the next generation of cells. 

• Proteins control most of what goes on within cells and within 
the body. 

• There are interactions among the cells of an organism; mole- 
cules from one cell affect what goes on inside other cells. 

• In plant cells, energy from sunlight is transformed into chemi- 
cal energy during photosynthesis; in plant and animal cells, 
the chemical energy stored in food molecules is released dur- 
ing digestion and produces heat. Some of this released energy 
is used to build new molecules. 

Organisms: 

• Separate parts of the body can communicate with one another 
using electrical or chemical signals. 

• Complex interacting systems include feedback that tends to 
produce cycles of activities within the body. 

• In organisms that reproduce sexually, each parent passes on 
one -half of its DNA information to each of its offspring; 
therefore, half of the DNA in each cell of an organism came 
from one parent, and half from the other parent. 

Ecology: 

• Interactions between living and nonliving components affect 
how ecosystems function as a whole. 

• A change in one component of an ecosystem affects other 
components of an ecosystem. These components in turn 
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react in a way that will restore the ecosystem to its original 
condition. 

• Often, changes in one component of an ecosystem will have 
effects on the entire system that are difficult to predict. 

• The size of a population and the rate of growth of a popula- 
tion are determined largely hy the survival rate, reproductive 
rate, and death rate of the organisms in the population. Pre- 
dictions about changes in the size or rate of growth of a popu- 
lation can he descrihed using mathematical models. 

Evolution: 

• Recombination and mutation are the raw materials for new 
traits upon which natural selection acts. 

• When the environment changes, different characteristics may 
become important for survival; that is, different adaptations 
are important for survival in different environments. 

• Some descendants are so different from other descendants 
that they can no longer breed with each other. 

Particularly at grade 12, students should be able to integrate in- 
formation from different concepts within the life, physical, and 
Earth sciences. They should understand how key concepts apply at 
different levels of biological organization (molecular, cellular, or- 
ganism, population, community, ecosystem, and biome) and how 
these concepts apply to current societal problems and are signifi- 
cant to the development of a variety of biotechnologies. They 
should be able to describe common misconceptions about natural 
phenomena and how these explanations are contrary to contempo- 
rary scientific explanations. Students should also understand the ef- 
fects of technologies created by humans on the life cycles of 
organisms and their effects on communities of plants and animals, 
including humans. In addition, students should have developed 
some familiarity with the historical development of key concepts in 
the life sciences. 
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Appendix C 

Examples of Themes by Grade Level 




Examples of Themes by Grade Level 



Systems 

S tudents should understand that systems are artificial construc- 
tions created hy people for certain purposes, such as gaining a 
better understanding of the natural world or designing an ef- 
fective technology. 

Understanding the construct of a system entails identifying and 
defining its boundaries, identifying its component parts, identifying 
the interrelations and interconnections among the component parts, 
and identifying the inputs and outputs of the system. 

Systems should be embedded in life science learning at the three 
grade levels in the following ways: 

Grade 4 

Systems should be approached at the level of organisms. Students 
should have broad and rich acquaintance with structure/function re- 
lationships as a precursor to a more thorough knowledge of organ 
systems by grade 8. Understanding examples of food chains and 
interdependencies among organisms, say, within an aquarium, are 
precursors to understanding complex systems. 

Grade 8 

Students should understand that an organism is made up of 
organ systems that have structure/function adaptations and 
interconnections among other organ systems. 

Interdependence of plants and animals in communities should be 
understood by grade 8: plants ->consumers->decomposers. Students 
should be able to explain specific examples such as purple loose- 
strife replacing cattails and the effects of the introduction of rabbits 
into Australia. 

Disease and health should be understood in systems terms. If a 
part of a system is put out of kilter by disease, for example, the 
whole system is affected. Taking drugs or smoking by an individual 
may have an impact on another system (organism); for example. 
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secondary smoke affects children of smoking parents or a fetus 
may be damaged by drugs. A measles vaccine taken or not taken 
by an individual affects the whole population of a region or even 
further, depending on migration patterns. If a specific animal or 
plant population becomes unhealthy (for example, fish poisoned, 
raccoons diseased, or species of grass infected by vims), the food 
chain and, therefore, the rest of the community are affected. 

Grade 12 

Ecosystems should be understood in their full complexity, includ- 
ing interrelationships of plants and animals with one another and 
with the physical components of a system. Students also need to 
recognize the effects of human activity on ecosystems and the limi- 
tations on human activity imposed by natural systems. 

At this level, the cell should be understood both as a system in 
itself and as a component of a system. 

Patterns of Change 

Patterns of Change is a particularly valuable theme in the life 
sciences because a conceptual understanding of patterns of change 
can be developed in the context of several different levels in the hi- 
erarchy of biological organization. At the cellular/organismal level, 
the primary patterns of change are the growth and development 
that occur throughout the life of organisms. At the population level, 
the primary patterns of change are the changes in population 
growth over relatively short periods of time and the evolutionary 
changes that occur over longer periods of time. At the community/ 
ecosystem level, the primary patterns of change are those that in- 
volve the nonliving and living components of ecosystems during 
the process of succession. Patterns of change may be linear, or they 
may be cyclical; for example, many of the patterns of change that 
occur within cells are related to homeostasis, in which a change 
leads to feedback reactions that result in a return to conditions that 
existed before the change. An understanding of cyclical patterns of 
change can also be developed in the context of ecosystems (nutri- 
ent cycles) and organisms (life cycles). 

Regardless of the context in which an understanding of the 
Patterns of Change theme is developed, students should be able 
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to recognize patterns of similarity and difference, to recognize how 
these patterns change over time, and to transfer their understanding 
of a familiar pattern of change to a new, unfamiliar situation. 

To understand the conceptual basis for the Patterns of Change 
theme, students must begin to develop an understanding of major 
ideas by the 4th grade, continue to develop their understanding 
through the 8th grade, and integrate their understanding with their 
knowledge in the 12th grade. 

Grade 4 

Understanding patterns of change at the organismal level: 

• life cycles (including growth and metamorphosis) 
Understanding patterns of change at the population level: 

• concept of biotic potential, birth rates, and survival rates 

• diversity of many types of plants and animals (an important 
preconcept for the understanding of evolution) 

• variation within species (focus on humans, dogs, and cats) 

Understanding patterns of change at the community/ecosystem 
level: 

• food chains (also important for the systems theme) 

A more general understanding involves the notion that every- 
thing changes, sometimes quickly and sometimes slowly, and that 
changes may be too rapid or too slow to observe directly. 

Grade 8 

Understanding patterns of change at the organismal level: 

• growth, development, and reproduction of the human 
organism 

• homeostasis of body systems 

Understanding patterns of change at the population level: 

• adaptation and natural selection, including learned and in- 
stinctive behavior 
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• variation and similarity among many different organisms, in- 
cluding humans 

Understanding patterns of change at the community/ecosystem 
level: 

• food wehs (also part of the systems theme) 

• environmental effects of human activity (also part of the sys- 
tems theme) 

More general understandings involve the following knowledge: 
changes in quantity usually have natural limits, hut changes in form 
in which each form arises from a previous one can produce an un- 
limited variety; the rate of change may he as interesting as the 
change itself; and trends can he steady, accelerated, approach a 
limit gradually, or have a highest or lowest value. 

Grade 12 

Students should have acquired an understanding of the following 
concepts and developed the ability to integrate them into the Pat- 
terns of Change theme. 

Understanding patterns of change at the cellular/organismal level: 

• growth and development of cells, including an understanding 
of the importance of mitosis and meiosis 

Understanding patterns of change at the population level: 

• patterns of evolution, mechanisms for evolution, consequences 
of evolution (such as speciation and diversity through time), 
and evidence for evolution 

Understanding patterns of change at the community/ecosystem 
level: 

• nutrient cycles and the impact of human activity on those 
cycles 

• succession, both natural and as a result of human disturbance 

More general understandings entail knowing that trends, cycles, 
and randomness can occur at the same time; randomness may make 
it hard to see trends or cycles; randomness sometimes may look like 
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a trend or cycle; feedback in systems, often an influence that reacts 
against change, tends to produce cycles; changes that follow precise 
rules from one moment to the next may still be unpredictable in the 
long run; and the environment in which any one change occurs is 
usually changing also, and each affects the other. 

Models 

Models of objects and events in nature can be used as approaches 
to understanding. As such, they have limits and involve simplifying 
assumptions but also possess generalizability and, sometimes, pre- 
dictive power. Models are composed of groups of interrelated con- 
cepts selected to represent the interrelations of objects or events in 
nature or in the laboratory. Models need not be deemed correct to 
be useful but may represent attempts to help identify relevant vari- 
ables to build evermore useful representations. 

Models may be conceptual and consist of word descriptions or 
drawings. Models can also be mathematical, consisting of equations 
or other formal representations. Finally, physical models consist of 
physical objects that possess or represent some characteristics of 
the real thing. 

The solar system is often modeled conceptually in the classroom 
by describing the planets as huge balls moving about an even larg- 
er Sun. A mathematical model of the solar system should include 
quantitative descriptions of the gravitational forces between the 
planets and the Sun as determined by their masses and distances 
from one another, and might include the shape of a planet’s orbit as 
being elliptical. And finally, a physical model of the solar system 
might consist of a series of scale-size balls placed at appropriate 
distances throughout a room or hallway. 

Other examples from the Earth and physical sciences include 
models of shorelines and continental plates as well as stick-and- 
ball models of molecules. Physical models, such as those of the 
eye, leaf, and human torso, have been used in the life sciences for 
decades. Experiments with animals serving as models of human 
beings have been used to understand the effects of medical treat- 
ments that might be useful in preventing human diseases and bacte- 
ria have been used to model population growth and decay. 
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Similarly, conceptual models are common in both the biological 
and physical sciences. The simplified treatment of photosynthesis, 
the stages of meiosis and mitosis, accounting for an electrical cur- 
rent in a “water flow” analogy, and the characterization of gas 
molecules as bouncing balls are examples of commonly used con- 
ceptual models. 

Mathematical models such as the gas laws and Newton’s laws of 
motion are major components of the physical sciences. In addition, 
some mathematical models, such as Mendef s laws, have been part 
of the biological sciences for most of this century, whereas the 
Hardy-Weinberg formulation for describing ecosystems mathemati- 
cally has become part of introductory biological knowledge more 
recently. 

Models often serve as prototypes in technology, and in that case 
may be full-sized representations of the final product. However, 
models can be used to test the workings of technology without 
costly investments in full-scale objects. Small boats and airplanes 
are tested in tanks and wind tunnels before their full-size counter- 
parts are built. In this way, many experiments can be tested inex- 
pensively to optimize the design. 

Models can be easily developed as a theme and can be linked to 
the immediate experiences of children because they have grown up 
with a variety of toys. Children readily understand that most toys 
are models that look like the real objects (such as cars, airplanes, 
babies, and animals) but do not possess all the attributes of those 
objects. Many of these toys are models, sometimes scale models, of 
objects from the natural world. For example, models of dinosaurs 
enable children to develop ideas about what these creatures were 
like. 

The models theme has been selected because of the importance of 
enabling students to distinguish the idealizations of models from the 
phenomena themselves. Students need to understand that a model of 
the human eye does not represent all aspects of human eyes as they 
occur in human organisms. The model is a simplification, leaving 
unrepresented many important variations in human eye structure, yet 
the simplification has utility in illuminating some features of the eye 
and enables new questions about the eye to be generated. 
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Students need to understand the limitations and simplifying as- 
sumptions that underlie the varied models used in the natural sci- 
ences. For example, beliefs that models are replicas of real objects 
or events can negate the critical concept of variation that many 
models do not take into account. Although generalized models, 
such as a generalized graph of growth in populations, are useful, 
they are not to be confused with a graph of the growth of a particu- 
lar organism or population or with a graph of data from a single 
experiment. 

Grade 4 

At this level, models should be identified by students as repre- 
sentations of objects or events. Students can examine both concep- 
tual and physical models in terms of how they are like and not like 
the object or event being represented. Examples can be models of 
insects, seeds, leaves, and other physical objects. These models and 
others in the sciences can be linked to children’s experiences with 
scale models of cars, dinosaurs, doll furniture, and so forth. 

Grade 8 

Students should have knowledge of both conceptual and physi- 
cal models and their uses and limitations. For example, when asked 
to illustrate their understanding of vertebrate structure and function 
with models of skeletons of different vertebrates, students need to 
be aware of variations in real skeletons and the generalized nature 
of the replicas. 

Grade 12 

Mathematical, physical, and conceptual models should be famil- 
iar to students beyond grade 8. It is appropriate to assess students’ 
ability to formalize the concept of models and their uses and limi- 
tations in the natural sciences and in technology. 
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Appendix D 

Science Content Outlines (Excerpts*) 



* Complete content outlines appear in the Specifications Document 




Science Content Outlines (Excerpts) 



Grade 4 — Earth Science 

A. Solid Earth (lithosphere) 

1. Composition of the Earth: 

• Students can classify substances such as soil, sand, or rock. 

• Students can identify common geographic features of 
landscapes. 

2. Eorces that alter the Earth’s surface: 

• Students can describe/explain basic facts about major features 
of the Earth’s surface and natural changes in those features 
(for example, volcanoes, glaciers): 

— Students can predict the effects of weathering (for example, 
rain and wind on sand piles, mud piles, rock). 

— Students can describe the relative difference in time it takes 
to erode a sand pile, a mud pile, and a rock pile ( Concep- 
tual Understanding, Patterns of Change). 

— Given a picture, topographical map or globe, or word de- 
scription of a major Earth feature (for example, canyon, 
mountain range. Great Eake, cavern, island), students can 
identify a geologic force that contributed to producing that 
feature (Conceptual Understanding, Models). 

3. Rocks: Their formation, characteristics, and uses: 

• Students can identify common rocks and minerals and can ex- 
plain how we can investigate what they are made of and how 
they form: 

— Students can classify rock samples according to color, tex- 
ture, or other identifying properties (Scientific Investiga- 
tion, Nature of Science). 

— Students can explain that molten rock comes out of volca- 
noes, hardens, and becomes part of the landscape ( Concep- 
tual Understanding, Patterns of Change). 
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4. Soil: its changes and uses: 

• Students know some facts about the composition of soil. 

• Students can separate soil samples into component parts 
(Scientific Investigation, Nature of Science, Systems). 

• Students recognize that plants grow in soil and that soil pro- 
vides both nutrients and support for the plant. 

• Students can classify and relate major solid types of soil (for 
example, clay, sand, loam, subsoil) to their ability to support 
plant growth; that is, students can identify/predict the major 
plant types likely to grow in those soils ( Conceptual Under- 
standing, Nature of Science). 

5. Resources from the Earth used hy humaukiud: 

• Students can identify Earth resources used in everyday life. 

• Students can identify common uses of rock in the human en- 
vironment (for example, buildings, roads, walls) (Practical 
Reasoning, Nature of Technology). 

• Students can explain/identify that gasoline is processed from 
oil, which is pumped from the Earth (Practical Reasoning, 
Nature of Technology). 

Grade 8 — Life Science 
A. Cells aud their fuuctious 
1. Cells: 

• Students can describe their observations of cells under the 
microscope: 

— Students can demonstrate the use of a microscope to exam- 
ine a tissue, plant, or animal and to differentiate between 
plant and animal cells (for example, students can look at an 
animal cell and a plant cell and notice that an animal cell is 
flexible and a plant cell is not) ( Scientific Investigation, 
Systems); 

— Students can look at pond water through a microscope and 
describe outstanding features/activities of the protista they 
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see (for example, locomotion, nutrition, excretion) ( Scien- 
tific Investigation). 

— Students can observe diatoms and try to distinguish as 
many features as possible (Scientific Investigation). 

• Students can explain, in a general way, the advantages of 
cellular interdependence versus independence (for example, 
multicellular animals versus single-celled animals). 

• Students can describe, in general terms, the difference be- 
tween asexual and sexual reproduction in cells and the advan- 
tages and disadvantages of each (the stages of mitosis are 
not to be tested). 

B. Organisms 

1. Reproduction, growth, and development: 

• Students can describe growth, development, and reproduction 
of the human organism: 

— Students can identify the age ranges at which human be- 
ings go through common stages of development (for ex- 
ample, can recognize their parents; can learn to walk, talk, 
and socialize; can conceive or give birth) (Conceptual 
Understanding, Patterns of Change). 

— Students can identify the changes human beings undergo 
at puberty and can explain their functions ( Conceptual 
Understanding, Patterns of Change). 

— Students can, in simple terms, describe changes in human 
embryo development and the effects of environmental in- 
fluences such as smoking, drugs, disease, and the mother’s 
diet on the development of the embryo ( Conceptual Under- 
standing, Patterns of Change ). 

2 . Life cycles: 

• Students can identify some major influences on the human 
life cycle (for example, diet, disease): 

— Students can discuss the influence of diet and food avail- 
ability on human life cycles worldwide (Practical Reason- 
ing, Patterns of Change). 
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— Students can explain that microorganisms can cause dis- 
ease and can identify some common diseases caused by 
microorganisms such as bacteria, viruses, protista (differ- 
ences between viruses and bacteria are not to be tested) 

(Conceptual Understanding). 

— Students can describe the immune system of animals 
as helping the animal fight disease and as controlled, in 
part, by the white blood cells in the body ( Conceptual 
Understanding ). 

3. Functions and interactions of systems within organisms: 

• Students are aware that, although different systems of the 
body have different functions, the functioning of each system 
affects other systems (for example, students can describe/ 
identify major organ systems of the human body, state their 
major functions, and describe some of their interactions): 

— Students can describe the primary tissues of the body (for 
example, blood, lymph, muscle) and relate the special char- 
acteristics of each to its function ( Conceptual Understand- 
ing, Systems). 

— Students can distinguish cells from other structures under 
the microscope (for example, between an onion cell and a 
salt crystal) (Scientific Investigation, Systems). 

— Students can describe how two or more organs of the body 
work together to perform a function (for example, the heart 
and lungs working together in respiration) ( Conceptual 
Understanding, Systems). 

• Students demonstrate an understanding of the functions and 
interactions of organ systems to maintain a stable internal en- 
vironment that can resist disturbances from within or without 
(homeostasis). 
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Grade 12 — Physical Science 

A. Matter and its transformations 

1. Diversity of matter (materials): Classification and types, 
particulate nature of matter, conservation of matter: 

• Students can distinguish/classify objects, both regular and 
irregular; pure substances, both elements and compounds; 
and mixtures, both homogeneous (solutions, liquids, gases) 
and nonhomogeneous. 

• Students can describe, measure, and compare substances in 
terms of mass, volume, and density/specific gravity: 

— Given a substance of unknown volume or weight and ap- 
propriate laboratory equipment, students can determine its 
specific gravity (Scientific Investigation). 

— Given an irregular solid and the appropriate laboratory 
equipment, students can determine the density of an object 
(Scientific Investigation). 

— Students can offer a simplified distinction between weight 
and density (Conceptual Understanding). 

• Students can identify evidence that matter is composed of 
tiny particles (for example, atoms, molecules) and that the 
particles are in motion (kinetic molecular theory). 

• Students can define, describe, and contrast physical, chemi- 
cal, and nuclear changes in molecular terms. Given various 
examples of changes in materials, students can distinguish 
among chemical, physical, and nuclear changes ( Conceptual 
Understanding, Patterns of Change). 

• Students can discuss the conservation of matter in physical, 
chemical, and nuclear changes (can also be tested under 
temperature and states of matter, or energy and its 
transformations). 

2. Temperature and states of matter (physical changes): 

• Students can discuss/identify the relationship of physical 
states of matter to molecular energy: 
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— Students can associate energy states with molecular motion 
(Conceptual Understanding). 

— Students can discuss/identify the energy transfers involved 
in the change of phase from solid to liquid to gas and the 
reverse (also tested under energy and its transforma- 
tions) (Conceptual Understanding, Patterns of Change). 

• Students can discuss/identify the relationship of physical 
changes in substances (for example, melting, boiling, thermal 
expansion and contraction, compression and expansion under 
pressure, increase or decrease in density) to changes in the 
structural organization of the atoms or molecules of which 
they are composed: 

— Students can explain how antifreeze solutions work to 
prevent freezing of water in car radiators (Practical 
Reasoning, Systems). 

— Students can explain why NaCl (sodium chloride/salt) is 
added to ice when making ice cream (Practical Reasoning). 

3. Properties and uses of materials: Modifying properties, 

synthesis of materials with new properties: 

• Students can relate the physical properties (for example, com- 
pressibility, structural rigidity) of pure substances in solid, 
liquid, and gaseous states to the structural organization of 
particles in the substance and their freedom of motion. Stu- 
dents can explain/identify that the molecules in a crystal are 
arranged in a regular pattern that gives the crystal rigidity and 
causes it to take a simple geometric shape ( Conceptual Un- 
derstanding, Models). 

• Students can examine/utilize useful properties of materials: 

— Given an unknown liquid and a universal indicator with 
a chart, students can determine the pH of the liquid ( Scien- 
tific Investigation). 

— Given an unknown marking pen, students can use paper 
chromatography to identify the brand of marking pen 
from among several other brands ( Scientific Investigation, 
Nature of Science). 
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• Students can describe how common artificial materials are 
made, recognizing that substances can be designed to have cer- 
tain properties and that the addition of relatively small amounts 
of some substances can significantly alter the properties. 

• Students can describe how common artificial materials are 
disposed of or recycled and can discuss the technological and 
environmental issues involved in these processes. 

4. Resource management: 

• Students can discuss scientific, technological, and social is- 
sues involved in resource management. 

• Students can discuss the issue of worker safety in manufac- 
turing processes that involve poisonous chemicals (Practical 
Reasoning, Nature of Technology). 
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Appendix E 

Science Committees and Project Staff 




NAEP Science Project Team 



staff 

Ramsay W. Selden 

Director 

State Education Assessment 
Center 

Council of Chief State School 
Officers 

Richard C. Clark 

Consensus Coordinator 
Council of Chief State School 
Officers 

Julia H. Mitchell 

Associate Research Scientist 
American Institutes for 
Research 

Senta A. Raizen 

Director 

National Center for Improving 
Science Education 

Bonnie L. Verrico 

Administrative Assistant 
Council of Chief State School 
Officers 

Steering Committee 
Members 

William O. Baker, Retired 

AT&T Bell Eahoratories 
Murray Hill, New Jersey 

Mary Louise Bellamy 

Education Director 
National Association of 
Biology Teachers 
Reston, Virginia 



Frank Betts 

Director 

Curriculum Technology Center 
Association for Supervision 
and Curriculum Development 
Alexandria, Virginia 

William B. Campbell 

Executive Director 
National Industry Council for 
Science Education 
College Park, Maryland 

Glenn A. Croshy 

Professor 

Chemistry Department 
Washington State University 
Pullman, Washington 

Gerald Difford 

Executive Director 
Colorado Association of 
Schools 

Englewood, Colorado 

Janice Earle 

Director 

Center on Educational Equity 
National Association of State 
Boards of Education 
Alexandria, Virginia 

John Fowler 

Director 

Triangle Coalition for Science 
and Technology 
College Park, Maryland 
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Johnnie Hamilton 

Principal 

Franklin Intermediate School 
Chantilly, Virginia 

Elam Hertzler, Retired 

Secretary’s Commission on 
Achieving Necessary Skills 
Washington, D.C. 

Ann Kahn, Former President 

Parent Teachers Association 
Fairfax, Virginia 

Douglas Lapp 

Director 

National Science Resources 
Center 

Smithsonian Institution 
Washington, D.C. 

John Layman 

Director 

Science Teaching Center 
University of Maryland 
College Park, Maryland 

Harold Pratt 

Executive Director 
Science and Technology 
Management 
Jefferson County Schools 
Golden, Colorado 

Douglas Reynolds 

Chief 

Bureau of Science Education 
State Department of Education 
Albany, New York 

Bella Rosenberg 

Assistant to the President 
American Eederation of 
Teachers 

Washington, D.C. 



Jane Sisk 

Biology Teacher 

Calloway County High School 

Murray, Kentucky 

Judith Torney-Purta 

Department of Human 
Development 
University of Maryland 
College Park, Maryland 

Planning Committee 
Members 

Andrew Ahlgren 

Associate Director 
Project 2061 

American Association for the 
Advancement of Science 
Washington, D.C. 

Bill Aldridge 

Executive Director 
National Science Teachers’ 
Association 
Washington, D.C. 

J. Myron Atkin 

Professor 

School of Education 
Stanford University 
Stanford, California 

Joan Boykoff Baron 

Assessment Coordinator 
Connecticut Common Core of 
Eearning 

Bureau of Evaluation and 
Student Assessment 
Connecticut Department of 
Education 

Hartford, Connecticut 
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Audrey Champagne 

Professor 

School of Education 
State University of New York 
Albany, New York 

Sally Crissman 

Lower School Head/Science 
Teacher 

Shady Hill School 
Cambridge, Massachusetts 

Edmund W. Gordon 

Professor 

Department of Psychology 

Yale University 

New Haven, Connecticut 

Henry Heikkinen 

Director 

M.A.S.T. Center, University of 
Northern Colorado 
Greeley, Colorado 

George Hein 

Professor 
Lesley College 
Cambridge, Massachusetts 



Joseph L. Premo 

Science Consultant 
Minneapolis Public Schools 
Minneapolis, Minnesota 

Senta A. Raizen 

Director 

National Center for Improving 
Science Education 
Washington, D.C. 

James Robinson, Retired 

Curriculum and Evaluation 
Boulder Valley Schools 
Boulder, Colorado 

Thomas P. Sachse 

Manager 

Math and Science Unit 
California Department of 
Education 

Sacramento, California 

Gary E. Skaggs 

Test Development Analyst 
Office of Research and 
Evaluation 

Eairfax County Public Schools 
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